Data Sources, Model, and Pipeline
Technical detail supporting the submitted proposal
This page expands on the methodology summarized in the proposal. It covers the two data sources the pipeline draws on, the two-phase construction process, and the resulting data model.
Data sources
The pipeline draws on two sources to build the initial dataset.
Funding records released by federal and provincial funding agencies. These include project titles, plain-language summaries, names of principal investigators and institutional hosts, and partner organizations. Funding records are the primary way an initiative first enters the dataset, since a funded project is identifiable independent of whether it has produced any publications yet.
OpenAlex, queried in two ways: by a set of metaresearch-related topic IDs drawn from OpenAlex’s own topic taxonomy, and by authors’ Canadian affiliations. Where a matching work has full text available, it is processed with GROBID, an open-source tool for parsing structured data from research articles, to extract information typically found in acknowledgements and funding disclosure sections: funding sources and relationships with umbrella organizations that supported the work.
Two-phase construction
The directory is constructed in two phases.
Phase 1: identification and collection
The first phase identifies, locates, and assembles information describing each initiative, producing a base dataset that is then enriched.
Funding records are enriched by locating each initiative’s official web pages and appending rosters of connected individuals. In parallel, OpenAlex records are processed as described above.
Names of individuals and initiatives logged from each source are matched by string similarity and by other identifiers where available (ORCID, email address), and matches are merged into a single record. Records that don’t match anything else prompt a manual query to locate additional information.
The outcome of this phase is a comprehensive dataset comprising: metaresearch initiatives, the works deriving from them, the people who produced that work, their institutional and disciplinary homes, and the sources from which they obtained financial support.
Phase 2: analysis
Each record from Phase 1 is explicitly linked to textual resources connected to it, including project web pages and scholarly outputs. The second phase analyzes these materials to determine specific aspects of metascience associated with each initiative.
Topic modelling and targeted LLM queries are applied to the titles, abstracts, and full texts (where available) of each scholarly output, and to initiatives’ own descriptions of their objectives and activities as documented on their websites. Topics derive from how each initiative identifies itself, its methods, and its disciplinary orientation.
These topics may also be fitted into more established taxonomies, including those of Peterson and Panofsky (2023) and Rubin (2026); these are appended to initiative records as tags and categories, respectively.
Data model
Phase 1 and Phase 2 together populate three kinds of information, linked to one another:
- Initiatives — the anchor record for each identified effort: its name, institutional or funding host, and connected textual resources (web pages, scholarly outputs).
- People — individuals connected to an initiative, drawn from principal investigator and roster information, with institutional and disciplinary affiliations and matched across sources by identifier or string similarity.
- Works — scholarly outputs and other textual resources deriving from an initiative, carrying the tags and categories assigned during Phase 2’s topic modelling and taxonomy fitting.
References
Peterson, David, and Aaron Panofsky. 2023. Metascience as a Scientific Social Movement. Minerva 61 (2): 147–74. https://doi.org/10.1007/s11024-023-09490-3.
Rubin, Mark. 2026. Six Types of Metascience. Blog post. Critical Metascience, June 17. https://markrubin.substack.com/p/contained-experiments-reproducibility.