Ethnicity ontologies#

lamindb provides access to the following public protein ontologies through lnschema-bionty:

  1. Human Ancestry Ontology

Here we show how to access and search ethnicity ontologies to standardize new data.

Setup#

!lamin init --storage ./test-ethnicity --schema bionty

✅ saved: User(uid='DzTjkKse', handle='testuser1', name='Test User1', updated_at=2023-12-22 11:24:41 UTC)
✅ saved: Storage(uid='ZAI9vHXr', root='/home/runner/work/lamin-usecases/lamin-usecases/docs/test-ethnicity', type='local', updated_at=2023-12-22 11:24:41 UTC, created_by_id=1)

💡 loaded instance: testuser1/test-ethnicity
💡 did not register local instance on hub

import lnschema_bionty as lb
import pandas as pd

# adds an entry "human" into an empty instance
lb.settings.organism = "human"

💡 loaded instance: testuser1/test-ethnicity

Bionty objects#

Let us create a public knowledge accessor with bionty(), which chooses a default public knowledge source from BiontySource. It’s a Bionty object, which you can think about as a less-capable registry:

ethnicity_bt = lb.Ethnicity.bionty()
ethnicity_bt

Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

📖 Ethnicity.df(): ontology reference table
🔎 Ethnicity.lookup(): autocompletion of terms
🎯 Ethnicity.search(): free text search of terms
✅ Ethnicity.validate(): strictly validate values
🧐 Ethnicity.inspect(): full inspection of values
👽 Ethnicity.standardize(): convert to standardized names
🪜 Ethnicity.diff(): difference between two versions
🔗 Ethnicity.ontology: Pronto.Ontology object

As for registries, you can export the ontology as a DataFrame:

df = ethnicity_bt.df()
df.head()
name definition synonyms parents
ontology_id
HANCESTRO:0002 region Any Geographic Area Greater Than An Individual... geographical area []
HANCESTRO:0003 country A Collective Generic Term That Refers Here To ... None []
HANCESTRO:0004 ancestry category Population Category Defined Using Ancestry Inf... ancestral group []
HANCESTRO:0005 European Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004]
HANCESTRO:0006 South Asian Includes Individuals Who Either Self-Report Or... None [HANCESTRO:0008]

Unlike registries, you can also export it as a Pronto object via ethnicity_bt.ontology.

Look up terms#

As for registries, terms can be looked up with auto-complete:

lookup = ethnicity_bt.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.american

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["American"]

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = ethnicity_bt.lookup(ethnicity_bt.ontology_id)

If multiple entries are matched, they are returned as a list:

lookup.hancestro_0463

Ethnicity(ontology_id='HANCESTRO:0463', name='American', definition=None, synonyms=None, parents=array(['HANCESTRO:0566'], dtype=object))

Search terms#

Search behaves in the same way as it does for registries:

ethnicity_bt = lb.Ethnicity.bionty()
ethnicity_bt.search("American").head(3)
ontology_id definition synonyms parents __ratio__
name
American HANCESTRO:0463 None None [HANCESTRO:0566] 100.0
African American HANCESTRO:0568 None None [HANCESTRO:0016] 90.0
African American or Afro-Caribbean HANCESTRO:0016 Includes Individuals Who Either Self-Report Or... None [HANCESTRO:0004] 90.0

By default, search also covers synonyms:

ethnicity_bt.search("Caucasian").head(3)
ontology_id definition synonyms parents __ratio__
name
European HANCESTRO:0005 Includes Individuals Who Either Self-Report Or... Caucasian|white [HANCESTRO:0004] 100.0
Asian HANCESTRO:0008 Includes Individuals That Either Self-Report O... Asian unspecified [HANCESTRO:0004] 90.0
Asia HANCESTRO:0030 None None [] 90.0

Search another field (default is .name):

ethnicity_bt.search(
    "General characterisation of Ancestry of a population",
    field=ethnicity_bt.definition,
).head()
ontology_id name synonyms parents __ratio__
definition
General Characterisation Of The Ancestry Of A Population Or Individual HANCESTRO:0304 ancestry status None [] 85.245902
A Population With Increased Genetic Homogeneity And Reduced Genetic Variation Due To Cultural Or Geographic Isolation HANCESTRO:0290 genetically isolated population founder population|population isolate [HANCESTRO:0004] 42.857143
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Indian. HANCESTRO:0598 Singaporean Indian None [HANCESTRO:0487] 39.735099
Includes Individuals Who Either Self-Report Or Have Been Described By Authors As Singaporean Malay. HANCESTRO:0597 Singaporean Malay None [HANCESTRO:0007] 38.666667
Population Category Defined Using Ancestry Informative Markers (Aims) Based On Genetic/Genomic Data HANCESTRO:0004 ancestry category ancestral group [] 38.410596

Standardize ethnicity identifiers#

Let us generate a DataFrame that stores a number of ethnicity identifiers, some of which corrupted:

df_orig = pd.DataFrame(
    index=[
        "Mende",
        "European",
        "South Asian",
        "Arab",
        "This ethnicity does not exist",
    ]
)
df_orig
Mende
European
South Asian
Arab
This ethnicity does not exist

We can check whether any of our values are validated against the ontology reference:

validated = ethnicity_bt.validate(df_orig.index, ethnicity_bt.name)
df_orig.index[~validated]

❗ 1 term (20.00%) is not validated: This ethnicity does not exist

Index(['This ethnicity does not exist'], dtype='object')

Ontology source versions#

For any given entity, we can choose from a number of versions:

lb.BiontySource.filter(entity="Ethnicity").df()
uid entity organism currently_used source source_name version url md5 source_website updated_at created_by_id
id
50 clid Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro 2023-12-22 11:24:41.807896+00:00 1

When instantiating a Bionty object, we can choose a source or version:

bionty_source = lb.BiontySource.filter(
    source="hancestro", version="3.0", organism="human"
).one()
ethnicity_bt = lb.Ethnicity.bionty(bionty_source=bionty_source)
ethnicity_bt

Ethnicity
Organism: human
Source: hancestro, 3.0
#terms: 342

📖 Ethnicity.df(): ontology reference table
🔎 Ethnicity.lookup(): autocompletion of terms
🎯 Ethnicity.search(): free text search of terms
✅ Ethnicity.validate(): strictly validate values
🧐 Ethnicity.inspect(): full inspection of values
👽 Ethnicity.standardize(): convert to standardized names
🪜 Ethnicity.diff(): difference between two versions
🔗 Ethnicity.ontology: Pronto.Ontology object

The currently used ontologies can be displayed using:

lb.BiontySource.filter(currently_used=True).df()
uid entity organism currently_used source source_name version url md5 source_website updated_at created_by_id
id
1 zvGR Organism vertebrates True ensembl Ensembl release-110 https://ftp.ensembl.org/pub/release-110/specie... f3faf95648d3a2b50fd3625456739706 https://www.ensembl.org 2023-12-22 11:24:41.806211+00:00 1
4 TE9h Organism bacteria True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... ee28510ed5586ea7ab4495717c96efc8 https://www.ensembl.org 2023-12-22 11:24:41.806336+00:00 1
5 OZIG Organism fungi True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/fungi/releas... dbcde58f4396ab8b2480f7fe9f83df8a https://www.ensembl.org 2023-12-22 11:24:41.806371+00:00 1
6 W07m Organism metazoa True ensembl Ensembl release-57 http://ftp.ensemblgenomes.org/pub/metazoa/rele... 424636a574fec078a61cbdddb05f9132 https://www.ensembl.org 2023-12-22 11:24:41.806408+00:00 1
7 AVh3 Organism plants True ensembl Ensembl release-57 https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... eadaa1f3e527e4c3940c90c7fa5c8bf4 https://www.ensembl.org 2023-12-22 11:24:41.806473+00:00 1
8 MdBu Organism all True ncbitaxon NCBItaxon Ontology 2023-06-20 s3://bionty-assets/df_all__ncbitaxon__2023-06-... 00d97ba65627f1cd65636d2df22ea76c https://github.com/obophenotype/ncbitaxon 2023-12-22 11:24:41.806515+00:00 1
9 o36k Gene human True ensembl Ensembl release-110 s3://bionty-assets/df_human__ensembl__release-... 832f3947e83664588d419608a469b528 https://www.ensembl.org 2023-12-22 11:24:41.806550+00:00 1
11 VTEw Gene mouse True ensembl Ensembl release-110 s3://bionty-assets/df_mouse__ensembl__release-... fa4ce130f2929aefd7ac3bc8eaf0c4de https://www.ensembl.org 2023-12-22 11:24:41.806620+00:00 1
13 Uhnp Gene saccharomyces cerevisiae True ensembl Ensembl release-110 s3://bionty-assets/df_saccharomyces cerevisiae... 2e59495a3e87ea6575e408697dd73459 https://www.ensembl.org 2023-12-22 11:24:41.806686+00:00 1
14 000Q Protein human True uniprot Uniprot 2023-03 s3://bionty-assets/df_human__uniprot__2023-03_... 1c46e85c6faf5eff3de5b4e1e4edc4d3 https://www.uniprot.org 2023-12-22 11:24:41.806719+00:00 1
16 tD7O Protein mouse True uniprot Uniprot 2023-03 s3://bionty-assets/df_mouse__uniprot__2023-03_... 9d5e9a8225011d3218e10f9bbb96a46c https://www.uniprot.org 2023-12-22 11:24:41.806784+00:00 1
18 vqWI CellMarker human True cellmarker CellMarker 2.0 s3://bionty-assets/human_cellmarker_2.0_CellMa... d565d4a542a5c7e7a06255975358e4f4 http://bio-bigdata.hrbmu.edu.cn/CellMarker 2023-12-22 11:24:41.806850+00:00 1
19 ypPK CellMarker mouse True cellmarker CellMarker 2.0 s3://bionty-assets/mouse_cellmarker_2.0_CellMa... 189586732c63be949e40dfa6a3636105 http://bio-bigdata.hrbmu.edu.cn/CellMarker 2023-12-22 11:24:41.806883+00:00 1
20 2Zjk CellLine all True clo Cell Line Ontology 2022-03-21 https://data.bioontology.org/ontologies/CLO/su... ea58a1010b7e745702a8397a526b3a33 https://bioportal.bioontology.org/ontologies/CLO 2023-12-22 11:24:41.806915+00:00 1
21 4shh CellType all True cl Cell Ontology 2023-08-24 http://purl.obolibrary.org/obo/cl/releases/202... 46e7dd89421f1255cf0191eca1548f73 https://obophenotype.github.io/cell-ontology 2023-12-22 11:24:41.806948+00:00 1
25 LmWQ Tissue all True uberon Uberon multi-species anatomy ontology 2023-09-05 http://purl.obolibrary.org/obo/uberon/releases... abcee3ede566d1311d758b853ccdf5aa http://obophenotype.github.io/uberon 2023-12-22 11:24:41.807078+00:00 1
29 zMWv Disease all True mondo Mondo Disease Ontology 2023-08-02 http://purl.obolibrary.org/obo/mondo/releases/... 7f33767422042eec29f08b501fc851db https://mondo.monarchinitiative.org 2023-12-22 11:24:41.807209+00:00 1
33 cxPr Disease human True doid Human Disease Ontology 2023-03-31 http://purl.obolibrary.org/obo/doid/releases/2... 64f083a1e47867c307c8eae308afc3bb https://disease-ontology.org 2023-12-22 11:24:41.807339+00:00 1
35 2wto ExperimentalFactor all True efo The Experimental Factor Ontology 3.57.0 http://www.ebi.ac.uk/efo/releases/v3.57.0/efo.owl 2ecafc69b3aba7bdb31ad99438505c05 https://bioportal.bioontology.org/ontologies/EFO 2023-12-22 11:24:41.807405+00:00 1
37 3SSF Phenotype human True hp Human Phenotype Ontology 2023-06-17 https://github.com/obophenotype/human-phenotyp... 65e8d96bc81deb893163927063b10c06 https://hpo.jax.org 2023-12-22 11:24:41.807471+00:00 1
40 nwdt Phenotype mammalian True mp Mammalian Phenotype Ontology 2023-05-31 https://github.com/mgijax/mammalian-phenotype-... be89052cf6d9c0b6197038fe347ef293 https://github.com/mgijax/mammalian-phenotype-... 2023-12-22 11:24:41.807569+00:00 1
41 zAfB Phenotype zebrafish True zp Zebrafish Phenotype Ontology 2022-12-17 https://github.com/obophenotype/zebrafish-phen... 03430b567bf153216c0fa4c3440b3b24 https://github.com/obophenotype/zebrafish-phen... 2023-12-22 11:24:41.807601+00:00 1
43 p1co Phenotype all True pato Phenotype And Trait Ontology 2023-05-18 http://purl.obolibrary.org/obo/pato/releases/2... bd472f4971492109493d4ad8a779a8dd https://github.com/pato-ontology/pato 2023-12-22 11:24:41.807667+00:00 1
44 h0rU Pathway all True go Gene Ontology 2023-05-10 https://data.bioontology.org/ontologies/GO/sub... e9845499eadaef2418f464cd7e9ac92e http://geneontology.org 2023-12-22 11:24:41.807700+00:00 1
46 fxHJ BFXPipeline all True lamin Bioinformatics Pipeline 1.0.0 s3://bionty-assets/bfxpipelines.json a7eff57a256994692fba46e0199ffc94 https://lamin.ai 2023-12-22 11:24:41.807765+00:00 1
47 chfO Drug all True dron Drug Ontology 2023-03-10 https://data.bioontology.org/ontologies/DRON/s... 75e86011158fae76bb46d96662a33ba3 https://bioportal.bioontology.org/ontologies/DRON 2023-12-22 11:24:41.807798+00:00 1
48 7JhT DevelopmentalStage human True hsapdv Human Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/HSAPDV/11... 52181d59df84578ed69214a5cb614036 https://github.com/obophenotype/developmental-... 2023-12-22 11:24:41.807831+00:00 1
49 JIKv DevelopmentalStage mouse True mmusdv Mouse Developmental Stages 2020-03-10 http://aber-owl.net/media/ontologies/MMUSDV/9/... 5bef72395d853c7f65450e6c2a1fc653 https://github.com/obophenotype/developmental-... 2023-12-22 11:24:41.807863+00:00 1
50 clid Ethnicity human True hancestro Human Ancestry Ontology 3.0 https://github.com/EBISPOT/hancestro/raw/3.0/h... 76dd9efda9c2abd4bc32fc57c0b755dd https://github.com/EBISPOT/hancestro 2023-12-22 11:24:41.807896+00:00 1
51 rsbG BioSample all True ncbi NCBI BioSample attributes 2023-09 s3://bionty-assets/df_all__ncbi__2023-09__BioS... 918db9bd1734b97c596c67d9654a4126 https://www.ncbi.nlm.nih.gov/biosample/docs/at... 2023-12-22 11:24:41.807929+00:00 1 
!lamin delete --force test-ethnicity
!rm -r test-ethnicity

💡 deleting instance testuser1/test-ethnicity
✅     deleted instance settings file: /home/runner/.lamin/instance--testuser1--test-ethnicity.env

✅     instance cache deleted
✅     deleted '.lndb' sqlite file
❗     consider manually deleting your stored data: /home/runner/work/lamin-usecases/lamin-usecases/docs/test-ethnicity