Version history

{
  "cells": [
    {
      "source": "# GAP-10: CELLxGENE Census obs-schema inspection\n\n**Purpose:** Establish empirically which `obs` column names carry donor identity in ROSMAP and SEA-AD collections, and whether any shared external identifier (e.g., AMP-AD harmonized participant ID, NIAD brain bank accession) is present in both. This is the evidentiary foundation for `donor_resolver_v1` join-key selection.\n\n**Prior context (ticks 68–71):** Three ticks of EuropePMC/PubMed literature queries returned zero relevant results. No published ROSMAP_participant_id ↔ SEA-AD_donor_id equivalence table exists in indexed open-access literature. Strategy escalated to primary-data schema inspection.\n\n**Success criterion:** Identify ≥1 `obs` column present in both ROSMAP and SEA-AD Census slices whose value space overlaps, OR confirm that no direct join key exists and that an AMP-AD harmonized ID lookup is required as an intermediate step.",
      "cell_id": "c-1e0c10b2",
      "outputs": [],
      "cell_hash": "sha256:ede0a9d7debea43d3e3cbc3b28e17d735c5a78641caed0ac5a5116bf252814d3",
      "cell_type": "markdown",
      "execution_count": null
    },
    {
      "source": "import cellxgene_census\nimport pandas as pd\n\n# --- Step 1: Open Census and enumerate obs columns for ROSMAP and SEA-AD ---\nwith cellxgene_census.open_soma() as census:\n    human_obs = census[\"census_data\"][\"homo_sapiens\"].obs\n\n    # Pull a small ROSMAP slice: obs columns + sample of donor_id values\n    rosmap_obs = human_obs.read(\n        value_filter=\"dataset_id in ('8e10f1c4-8e98-41e5-b65f-8cd89a887122',\"\n                     \" 'f9ad5649-f372-43bd-b3b7-ab1fd0e342d8')\",\n        column_names=[\"soma_joinid\", \"dataset_id\", \"donor_id\",\n                      \"observation_joinid\", \"tissue_general\",\n                      \"cell_type\", \"assay\", \"suspension_type\"]\n    ).concat().to_pandas()\n\n    # Pull a small SEA-AD slice\n    seaad_obs = human_obs.read(\n        value_filter=\"dataset_id in ('de985818-285f-4f59-9dbd-d74968fddba3',\"\n                     \" 'c9d1c098-3e67-4a82-afc4-b8a7e1dfb504')\",\n        column_names=[\"soma_joinid\", \"dataset_id\", \"donor_id\",\n                      \"observation_joinid\", \"tissue_general\",\n                      \"cell_type\", \"assay\", \"suspension_type\"]\n    ).concat().to_pandas()\n\nprint('ROSMAP obs shape:', rosmap_obs.shape)\nprint('ROSMAP obs columns:', rosmap_obs.columns.tolist())\nprint('ROSMAP unique donor_id sample (first 20):', rosmap_obs['donor_id'].unique()[:20].tolist())\nprint()\nprint('SEA-AD obs shape:', seaad_obs.shape)\nprint('SEA-AD obs columns:', seaad_obs.columns.tolist())\nprint('SEA-AD unique donor_id sample (first 20):', seaad_obs['donor_id'].unique()[:20].tolist())",
      "cell_id": "c-12ab13e0",
      "outputs": [],
      "cell_hash": "sha256:6f4be7981941a0ab6094fbe4dc00ffb9b5799e42f141784594c44497d521a707",
      "cell_type": "code",
      "execution_count": null
    },
    {
      "source": "# --- Step 2: Enumerate ALL obs columns (full schema) via Census meta ---\nwith cellxgene_census.open_soma() as census:\n    full_col_df = census[\"census_data\"][\"homo_sapiens\"].obs.schema\n\nprint('Full obs schema field names:')\nfor field in full_col_df:\n    print(' ', field.name, '->', field.type)\n\n# --- Step 3: Cross-cohort column overlap ---\nrosmap_cols = set(rosmap_obs.columns.tolist())\nseaad_cols = set(seaad_obs.columns.tolist())\nprint('\\nShared obs columns between ROSMAP and SEA-AD slices:')\nprint(sorted(rosmap_cols & seaad_cols))\nprint('\\nROSMAP-only columns:', sorted(rosmap_cols - seaad_cols))\nprint('SEA-AD-only columns:', sorted(seaad_cols - rosmap_cols))\n\n# --- Step 4: Test donor_id value-space overlap ---\nrosmap_donors = set(rosmap_obs['donor_id'].dropna().unique())\nseaad_donors = set(seaad_obs['donor_id'].dropna().unique())\nprint('\\nROSMAP unique donors:', len(rosmap_donors))\nprint('SEA-AD unique donors:', len(seaad_donors))\nprint('Direct donor_id overlap:', len(rosmap_donors & seaad_donors))\nprint('Sample ROSMAP donor_id values:', sorted(list(rosmap_donors))[:10])\nprint('Sample SEA-AD donor_id values:', sorted(list(seaad_donors))[:10])",
      "cell_id": "c-e2bb7ebd",
      "outputs": [],
      "cell_hash": "sha256:416c3ecf4c5f507db5b056f11401767f073062cf3e79c13873b3d1ff79d4e28e",
      "cell_type": "code",
      "execution_count": null
    }
  ],
  "metadata": {},
  "owner_ref": "persona-andy-hickl",
  "created_by": "persona-andy-hickl"
}