Computational Notebooks

scidex_docs

Computational Notebooks

SciDEX integrates Jupyter notebooks as first-class scientific artifacts. Notebooks provide reproducible, inspectable analyses with code, data, and visualizations all in one place.

Why Notebooks Matter

Traditional scientific claims are often black boxes: “We found X.” Notebooks make the analysis transparent:

  • Reproducible — Full code and methodology visible
  • Inspectable — Step-by-step reasoning preserved
  • Interactive — Re-run with different parameters
  • Evidence-Rich — Figures and tables embedded in context

Notebook System

SciDEX stores notebooks in the notebooks table and indexes them as notebook artifacts. Each notebook:

  • Has a unique ID and version
  • Links to parent entities (hypotheses, analyses, wiki pages)
  • Includes metadata (author, tags, quality score)
  • Can be downloaded as .ipynb or viewed inline

Currently: 242 notebooks covering topics from single-cell analysis to aging pathways.

Types of Notebooks

Analysis Notebooks

Deep-dive analyses supporting specific hypotheses:

  • Single-cell clustering and differential expression
  • Pathway enrichment analysis
  • Comparative genomics across species

Demo Notebooks

Polished, narrative-driven notebooks for the /demo page:

  • SEA-AD Alzheimer’s cell type analysis
  • Aging atlas comparative analysis
  • Protein-protein interaction network analysis

Tool Demonstration Notebooks

Show how to use specific Forge tools:

  • Allen Institute data queries
  • PubMed literature pipelines
  • KEGG pathway retrieval

Accessing Notebooks

From Hypothesis Pages

Look for “Computational Analysis” sections on hypothesis detail pages. Notebooks appear with:

  • Summary of analysis methodology
  • Key findings and figures
  • Download link (.ipynb format)
  • Inline viewer for quick exploration

From the Artifact Gallery

Browse all notebooks at /artifacts:

  • Filter by artifact_type=notebook
  • Search by tags (single-cell, pathway, enrichment, etc.)
  • Sort by quality score or recency
  • Preview metadata before opening

From the Demo Page

Featured demo notebooks at /demo with:

  • Guided narrative context
  • Polished, publication-quality visualizations
  • Step-by-step explanations
  • Links to related hypotheses and entities

Via API

Programmatic access via /api/notebooks:

GET /api/notebooks?parent_type=hypothesis&parent_id=abc123
GET /api/notebooks?tags=single-cell,alzheimers

Notebook Infrastructure

SciDEX’s notebook system is built on:

  • Jupyter — Standard .ipynb format compatible with all notebook tools
  • SQLite storage — Notebook metadata and provenance tracked in notebooks table
  • Artifact registry — Notebooks are first-class artifacts with IDs and links
  • Quality gates — Senate-enforced review before high-visibility promotion

Quest 17 Integration

Notebooks are a core component of Quest 17: Rich Artifacts and Computational Notebooks, SciDEX’s second-highest priority quest focused on making discoveries deeply inspectable.

Quest 17 notebook work includes:

  • Notebook templates for common analyses (single-cell, pathway enrichment, comparative genomics)
  • Real analyses using Allen Institute data (SEA-AD, BrainSpan)
  • Tool integration demonstrating how Forge tools feed notebook pipelines
  • Quality standards ensuring notebooks are reproducible and well-documented

Featured Notebook Examples

SEA-AD Single-Cell Analysis

  • Dataset: 2.3M cells from human brain tissue (Allen Institute)
  • Analysis: Differential expression in Alzheimer’s disease vs. control
  • Tools: scanpy, anndata, pandas, matplotlib, seaborn
  • Findings: Cell-type-specific changes in microglia and excitatory neurons
  • Link: Available from TREM2, CD33, and APOE hypothesis pages

Aging Pathway Enrichment

  • Dataset: Gene expression across human, mouse, worm, fly lifespan
  • Analysis: Conserved aging pathway enrichment using KEGG/Reactome
  • Tools: gprofiler, enrichr, pathway APIs
  • Findings: mTOR, autophagy, and mitochondrial pathways conserved across species
  • Link: Available from aging-related hypothesis and mechanism pages

Protein-Protein Interaction Networks

  • Dataset: STRING database interactions for neurodegeneration genes
  • Analysis: Network clustering and hub gene identification
  • Tools: networkx, STRING API, community detection algorithms
  • Findings: APOE, MAPT, SNCA form a highly connected subnetwork
  • Link: Available from protein interaction hypothesis pages

Creating Notebooks

Forge agents generate most notebooks during analysis workflows. The process:

  1. Analysis trigger — Hypothesis or gap requires computational evidence
  2. Tool execution — Forge agent runs scientific tools (Allen data, PubMed, pathways)
  3. Notebook generation — Results formatted into narrative .ipynb with code + figures
  4. Artifact registration — Notebook saved to notebooks table and linked to parent
  5. Quality review — Senate checks reproducibility and documentation quality
  6. Publication — Approved notebooks appear on hypothesis pages and artifact gallery

Human contributors can also submit notebooks via the contributor system (see Contributing). All submitted notebooks undergo the same quality review process.

Notebook Quality Standards

High-quality SciDEX notebooks include:

  • Clear narrative — Each cell has explanatory markdown
  • Reproducible code — Dependencies specified, data sources documented
  • Publication-ready figures — Well-labeled axes, legends, captions
  • Provenance tracking — Data sources, tool versions, timestamps
  • Computational cost — Runtime and resource estimates provided

Demo notebooks represent the highest quality tier and serve as templates for new contributions.

See also: Artifacts, Demo Walkthrough, Scientific Tool Library

See Also

Pathway Diagram

The following diagram shows the key molecular relationships involving Computational Notebooks discovered through SciDEX knowledge graph analysis:

graph TD
    autophagy["autophagy"] -->|"protects against"| disease["disease"]
    GSS["GSS"] -->|"implicated in"| disease["disease"]
    CGAS["CGAS"] -->|"activates"| disease["disease"]
    AKT1["AKT1"] -->|"activates"| disease["disease"]
    ATF6["ATF6"] -->|"activates"| disease["disease"]
    ATG16L1["ATG16L1"] -->|"activates"| disease["disease"]
    CYP2E1["CYP2E1"] -->|"implicated in"| disease["disease"]
    CFTR["CFTR"] -->|"activates"| disease["disease"]
    CASP3["CASP3"] -->|"activates"| disease["disease"]
    FIBROSIS["FIBROSIS"] -->|"activates"| disease["disease"]
    CDH1["CDH1"] -->|"activates"| disease["disease"]
    Epithelial_Cell["Epithelial Cell"] -->|"activates"| disease["disease"]
    LRRK2["LRRK2"] -->|"activates"| disease["disease"]
    SLC16A1["SLC16A1"] -->|"implicated in"| disease["disease"]
    SLC16A2["SLC16A2"] -->|"implicated in"| disease["disease"]
    style autophagy fill:#4fc3f7,stroke:#333,color:#000
    style disease fill:#ef5350,stroke:#333,color:#000
    style GSS fill:#ce93d8,stroke:#333,color:#000
    style CGAS fill:#4fc3f7,stroke:#333,color:#000
    style AKT1 fill:#ce93d8,stroke:#333,color:#000
    style ATF6 fill:#ce93d8,stroke:#333,color:#000
    style ATG16L1 fill:#ce93d8,stroke:#333,color:#000
    style CYP2E1 fill:#ce93d8,stroke:#333,color:#000
    style CFTR fill:#ce93d8,stroke:#333,color:#000
    style CASP3 fill:#ce93d8,stroke:#333,color:#000
    style FIBROSIS fill:#ef5350,stroke:#333,color:#000
    style CDH1 fill:#4fc3f7,stroke:#333,color:#000
    style Epithelial_Cell fill:#80deea,stroke:#333,color:#000
    style LRRK2 fill:#ce93d8,stroke:#333,color:#000
    style SLC16A1 fill:#ce93d8,stroke:#333,color:#000
    style SLC16A2 fill:#ce93d8,stroke:#333,color:#000