Merge pull request #181 from ufbmi/lang-20220311

Updated ODK to 1.3.0; deprecated and replaced 'role in human social p…

.github/workflows/docs.yml

@@ -0,0 +1,27 @@
+# Basic ODK workflow
+name: Docs
+
+# Controls when the action will run. 
+on:
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  build:
+    name: Deploy docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout main
+        uses: actions/checkout@v2
+
+      - name: Deploy docs
+        uses: mhausenblas/mkdocs-deploy-gh-pages@main
+        # Or use mhausenblas/mkdocs-deploy-gh-pages@nomaterial to build without the mkdocs-material theme
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          CONFIG_FILE: mkdocs.yaml
+

.github/workflows/qc.yml

@@ -19,7 +19,7 @@ jobs:
   ontology_qc:
     # The type of runner that the job will run on
     runs-on: ubuntu-latest
-    container: obolibrary/odkfull:v1.2.26
+    container: obolibrary/odkfull:v1.3.0
 
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:

docs/odk-workflows/EditorsWorkflow.md

@@ -23,7 +23,7 @@ Ensure that there is a ticket on your issue tracker that describes the change yo
 In your local environment (e.g. your laptop), make sure you are on the `main` (prev. `master`) branch and ensure that you have all the upstream changes, for example:
 
 ```
-git checkout master
+git checkout main
 git pull
 ```
 

docs/odk-workflows/ManageDocumentation.md

@@ -15,7 +15,7 @@ All the documentation is contained in the `docs` directory, and is managed in _M
 1. Open the `.md` file you want to change in an editor of choice (a simple text editor is often best). _IMPORTANT_: Do not edit any files in the `docs/odk-workflows/` directory. These files are managed by the ODK system and will be overwritten when the repository is upgraded! If you wish to change these files, make an issue on the [ODK issue tracker](https://github.com/INCATools/ontology-development-kit/issues).
 2. Perform the edit and save the file
 3. Commit the file to a branch, and create a pull request as usual. 
-4. If your development team likes your changes, merge the docs into master.
+4. If your development team likes your changes, merge the docs into main branch.
 5. Deploy the documentation (see below)
 
 ## Deploy the documentation

docs/odk-workflows/ReleaseWorkflow.md

@@ -13,8 +13,8 @@ These steps are outlined in detail in the following.
 Preparation:
 
 1. Ensure that all your pull requests are merged into your main (master) branch
-2. Make sure that all changes to master are committed to Github (`git status` should say that there are no modified files)
-3. Locally make sure you have the latest changes from master (`git pull`)
+2. Make sure that all changes to main are committed to Github (`git status` should say that there are no modified files)
+3. Locally make sure you have the latest changes from main (`git pull`)
 4. Checkout a new branch (e.g. `git checkout -b release-2021-01-01`)
 5. You may or may not want to refresh your imports as part of your release strategy (see [here](UpdateImports.md))
 6. Make sure you have the latest ODK installed by running `docker pull obolibrary/odkfull`
@@ -29,7 +29,7 @@ This will create all the specified release targets (OBO, OWL, JSON, and the vari
 
 ## Review the release
 
-1. (Optional) Rough check. This step is frequently skipped, but for the more paranoid among us (like the author of this doc), this is a 3 minute additional effort for some peace of mind. Open the main release (ont.owl) in you favourite development environment (i.e. Protege) and eyeball the hierarchy. We recommend two simple checks: 
+1. (Optional) Rough check. This step is frequently skipped, but for the more paranoid among us (like the author of this doc), this is a 3 minute additional effort for some peace of mind. Open the main release (omrse.owl) in you favourite development environment (i.e. Protege) and eyeball the hierarchy. We recommend two simple checks: 
     1. Does the very top level of the hierarchy look ok? This means that all new terms have been imported/updated correctly.
     2. Does at least one change that you know should be in this release appear? For example, a new class. This means that the release was actually based on the recent edit file. 
 2. Commit your changes to the branch and make a pull request
@@ -49,3 +49,21 @@ Once your [CI checks](ContinuousIntegration.md) have passed, and your reviews ar
 1. You can write whatever you want in the release title, but we typically write the date again. The description underneath should contain a concise list of changes or term additions.
 1. Click "Publish release". Done.
 
+## Debugging typical ontology release problems
+
+### Problems with memory
+
+When you are dealing with large ontologies, you need a lot of memory. When you see error messages relating to large ontologies such as CHEBI, PRO, NCBITAXON, or Uberon, you should think of memory first, see [here](https://github.com/INCATools/ontology-development-kit/blob/master/docs/DealWithLargeOntologies.md).
+
+### Problems when using OBO format based tools
+
+Sometimes you will get cryptic error messages when using legacy tools using OBO format, such as the ontology release tool (OORT), which is also available as part of the ODK docker container. In these cases, you need to track down what axiom or annotation actually caused the breakdown. In our experience (in about 60% of the cases) the problem lies with duplicate annotations (`def`, `comment`) which are illegal in OBO. Here is an example recipe of how to deal with such a problem:
+
+1. If you get a message like `make: *** [cl.Makefile:84: oort] Error 255` you might have a OORT error. 
+2. To debug this, in your terminal enter `sh run.sh make IMP=false PAT=false oort -B` (assuming you are already in the ontology folder in your directory) 
+3. This should show you where the error is in the log (eg multiple different definitions) 
+WARNING: THE FIX BELOW IS NOT IDEAL, YOU SHOULD ALWAYS TRY TO FIX UPSTREAM IF POSSIBLE
+4. Open `omrse-edit.owl` in Protege and find the offending term and delete all offending issue (e.g. delete ALL definition, if the problem was "multiple def tags not allowed") and save. 
+*While this is not idea, as it will remove all definitions from that term, it will be added back again when the term is fixed in the ontology it was imported from and added back in.
+5. Rerun `sh run.sh make IMP=false PAT=false oort -B` and if it all passes, commit your changes to a branch and make a pull request as usual.
+

docs/odk-workflows/RepoManagement.md

@@ -86,9 +86,23 @@ import_group:
       module_type: filter
 ```
 
-A ROBOT filter module is, essentially, importing all external terms declared by the your ontology (see [here](UpdateImports.md)] on how to declare external terms to be imported).
+A ROBOT filter module is, essentially, importing all external terms declared by the your ontology (see [here](UpdateImports.md)] on how to declare external terms to be imported). Note that the `filter` module does 
+not consider terms/annotations from namespaces other than the base-namespace of the ontology itself. For example, in the
+example of GO above, only annotations / axioms related to the GO base IRI (http://purl.obolibrary.org/obo/GO_) would be considered. This 
+behaviour can be changed by adding additional base IRIs as follows:
 
 
+```
+import_group:
+  products:
+    - id: go
+      module_type: filter
+      base_iris:
+        - http://purl.obolibrary.org/obo/GO_
+        - http://purl.obolibrary.org/obo/CL_
+        - http://purl.obolibrary.org/obo/BFO
+```
+
 If you wish to customise your import entirely, you can specify your own ROBOT command to do so. To do that, add the following to your repo config (`src/ontology/omrse-odk.yaml`):
 
 ```

docs/odk-workflows/SettingUpDockerForODK.md

@@ -1,5 +1,13 @@
 # Setting up your Docker environment for ODK use
 
-Memory 
+One of the most frequent problems with running the ODK for the first time is failure because of lack of memory. This can look like a Java OutOfMemory exception, 
+but more often than not it will appear as something like an `Error 137`. There are two places you need to consider to set your memory:
+
+1. Your src/ontology/run.sh (or run.bat) file. You can set the memory in there by adding 
+`robot_java_args: '-Xmx8G'` to your src/ontology/omrse-odk.yaml file, see for example [here](https://github.com/INCATools/ontology-development-kit/blob/0e0aef2b26b8db05f5e78b7c38f807d04312d06a/configs/uberon-odk.yaml#L36).
+2. Set your docker memory. By default, it should be about 10-20% more than your `robot_java_args` variable. You can manage your memory settings
+by right-clicking on the docker whale in your system bar-->Preferences-->Resources-->Advanced, see picture below.
+
+![dockermemory](https://github.com/INCATools/ontology-development-kit/raw/master/docs/img/docker_memory.png)
+
 
-etc

docs/odk-workflows/UpdateImports.md

@@ -4,7 +4,9 @@ This page discusses how to update the contents of your imports, like adding or r
 
 ## Importing a new term
 
-Importing a new term is split into to sub-phases:
+Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title "Using the Base Module approach".
+
+Importing a new term is split into two sub-phases:
 
 1. Declaring the terms to be imported
 2. Refreshing imports dynamically
@@ -18,7 +20,8 @@ There are three ways to declare terms that are to be imported from an external o
 
 #### Protege-based declaration
 
-This workflow is to be avoided, but may be appropriate if the editor _does not have access to the ODK docker container_.
+This workflow is to be avoided, but may be appropriate if the editor _does not have access to the ODK docker container_. 
+This approach also applies to ontologies that use base module import approach.
 
 1. Open your ontology (edit file) in Protege (5.5+).
 1. Select 'owl:Thing'
@@ -81,7 +84,7 @@ First, you navigate in your terminal to the ontology directory (underneath src i
 cd src/ontology
 ```
 
-Then, you regenerate the import that will now include any new terms you have added.
+Then, you regenerate the import that will now include any new terms you have added. Note: You must have [docker installed](SettingUpDockerForODK.md).
 
 ```
 sh run.sh make PAT=false imports/go_import.owl -B
@@ -104,3 +107,70 @@ If you wish to skip refreshing the mirror, i.e. skip downloading the latest vers
 ```
 sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B
 ```
+
+## Using the Base Module approach
+
+Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files.
+The idea is to only import axioms from ontologies that _actually belong to it_. 
+A base file is a subset of the ontology that only contains those axioms that nominally 
+belong there. In other words, the base file does not contain any axioms that belong
+to another ontology. An example would be this:
+
+Imagine this being the full Uberon ontology:
+
+```
+Axiom 1: BFO:123 SubClassOf BFO:124
+Axiom 1: UBERON:123 SubClassOf BFO:123
+Axiom 1: UBERON:124 SubClassOf UBERON 123
+```
+
+The base file is the set of all axioms that are about UBERON terms:
+
+```
+Axiom 1: UBERON:123 SubClassOf BFO:123
+Axiom 1: UBERON:124 SubClassOf UBERON 123
+```
+
+I.e.
+
+```
+Axiom 1: BFO:123 SubClassOf BFO:124
+```
+
+Gets removed.
+
+The base file pipeline is a bit more complex then the normal pipelines, because
+of the logical interactions between the imported ontologies. This is solved by _first 
+merging all mirrors into one huge file and then extracting one mega module from it.
+
+Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies.
+When we use the base pipelines, we
+
+1) First obtain the base (ususally by simply downloading it, but there is also an option now to create it with ROBOT)
+2) We merge all base files into one big pile
+3) Then we extract a single module `imports/merged_import.owl`
+
+The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml.
+
+To check if your ontology uses this method, check src/ontology/omrse-odk.yaml to see if `use_base_merging: TRUE` is declared under `import_group`
+
+If your ontology uses Base Module approach, please use the following steps: 
+
+First, add the term to be imported to the term file associated with it (see above "Using term files" section if this is not clear to you)
+
+Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). 
+```
+cd src/ontology
+```
+
+Then refresh imports by running
+
+```
+sh run.sh make imports/merged_import.owl
+```
+Note: if your mirrors are updated, you can run `sh run.sh make no-mirror-refresh-merged`
+
+This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and your one of the local devs should pick this up and run the import for you.
+
+Lastly, restart Protege, and the term should be imported in ready to be used.
+

docs/templates/dosdp.md

@@ -0,0 +1,3 @@
+# DOSDP documentation stub
+
+Do not overwrite, contents will be generated automatically,