[Clare72]

vfb-scRNAseq (the 17 VFBseq_ datasets that replace the FBlc loads) adds Sample, Assay and Clustering individuals. It also links Samples to FlyBase stocks, strains and genetic entities. Until now none of these had term info or queries. Three commits: term info, FindStocks for strains, then the queries with stage and tissue columns.

What changed

Term info

• Sample, Assay and Clustering pages are built by term_info_fallback when Solr has no document. A new ScRNAseqEntityTermInfoQueryIndexer reuses the indexer's Cluster query on any VFBseq_ Individual. A Clustering's page doesn't list its Samples inline; SamplesInClustering lists them.
• Related-individual links fall back to the end of the IRI when there's no short_form, so the links aren't dead.
• Licence records with no licence node are dropped (_real_licenses). Stock pages showed an empty Licence row.

New queries (registered in ha_api.QUERY_TYPE_MAP; reference entry 31a–i)

• ClustersInClustering, SamplesInClustering, AssaysInClustering.
• ClusteringsWithSample, which includes other Datasets that reuse the Sample.
• ClusteringsInDataset, SamplesInDataset, AssaysInDataset.
• ScRNAseqSamplesFrom and ScRNAseqClustersFrom, on nodes labelled hasScRNAseqSample (stocks, strains, alleles, splits). They follow BAO is derived from and GENO has_genotype.

Existing queries

• anatScRNAseqQuery shows cell counts.
• scRNAdatasetData: the count now uses the same pattern as the rows, Clusters with a cell type. It used to count every Individual from the Dataset, including Samples, Assays and Clusterings.
• Stage and tissue columns: stage and tissue are curated on Samples. A Cluster has its own stage only where it was determined from its expression (Calderon 2022: 129 clusters), and it never has its own tissue. The columns keep the two apart:
  • Clusterings tables: "Sample stages", "Sample tissues".
  • Clusters within one Clustering: "Cluster stage (from expression)".
  • Cluster tables that span Clusterings (anatScRNAseqQuery, expressionCluster, scRNAdatasetData, ScRNAseqClustersFrom): all three columns.
  • Collapsing: sample stages collapse along FBdv substage_of to the narrowest stage covering each period. They never go above embryonic, larval, prepupal, pupal, pharate adult or adult stage. For example, Davie's eight adult ages show as adult stage.
  • Speed: Sample stages and tissues are fetched once per Clustering rather than per row. The cluster-stage edge is matched by type as well as short form, listing both existence_overlaps and existence overlaps. With this, gene → clusters stays at 0.3–0.6 s; an untyped match took 14 s.
• FindStocks for strains: it now handles FlyBase strains (FBsn) and lists the stock-centre stocks derived from the strain. FlyBase links them through strainprop derived_stock_strain_<centre>.

Testing

• New offline tests:
  • test_stage_collapse.py (stage collapsing);
  • test_stage_columns.py (turning raw query output into the stage and tissue columns, with the pdb lookups stubbed);
  • test_licenses.py (dropping null licences).
• New live tests:
  • test_scrnaseq_clustering.py (the 9 queries, their columns, limits and schemas);
  • test_stage_columns.py:
    • rows match headers in the three hand-built cluster tables;
    • a cluster's own stage shows;
    • FBdv substage_of ancestors resolve;
  • test_licenses.py: a stock page has no empty licence, and a dataset keeps its licence;
  • strain cases in test_flybase_stocks.py (DGRP-304; Canton-S at BDSC and Kyoto; DGRP-28, whose stock is gone).
• Updated: test_term_info_fallback.py.
• Ran test_scrnaseq_clustering, test_stage_columns, test_stage_collapse, test_licenses, test_transcriptomics, test_term_info_fallback, test_term_info_parity and test_flybase_stocks: 175 passed.
• Live tests ran against current pdb. That doesn't have the new data yet (Clustering → Sample links, strains, stock labels); to be checked again after the next data release.

[Robbie1977]

Reviewed, looks good.