Background

In our approach, we aim to avoid BigTech. We curate the queryable databases ourselves but welcome others to share their data catalogs. We utilize the DuckLake catalog from DuckDB and store the actual data as Parquet files on an S3-compatible object store. This separation of storage and compute helps us keep costs low. This catalog lists all the ORI data resoures, their tables and columns. also it holds information about the changes in time of all the data resources, like deleted, updated and added records, and schema changes. This alows you to 'time-travel' in the data, and not only use the last state. For example detecting when an article flips from open access to closed access. Compute: You can query the datasets directly from your browser (no login required)! When you query the databases, a portion of the requested data is transferred over HTTPS to your local machine where the SQL operations are performed. Larger data requests result in longer transfer times, and the speed of your local machine affects the query completion time. The size of your machine is up to you. At SURF, we also provide ready-made services for SQL computation, such as a Marimo notebook in a virtual machine on SURF Research Cloud or a Superset dashboard on a Kubernetes cluster. Below, you will see the ORI data resources we currently curate. (This overview was inspired by the ORION-DBS initiative.) We want to add the following datasets: OpenAIRE, OpenALEX, OpenAPC, ROR, Harvest metadata from CRISes, Harvest metdata from Repositories, Crossref, SURF Journal Catalogue, CWTS Leiden Ranking, ORCID, DOAJ, DOAB, OpenCitations, DataCite, PKP beacon.