CONP/PCNO

CONP Portal | FAQ

CONP | Data Access & Download | Data Upload | Tools & Pipelines | DataLad

I. CONP

What is CONP?

The Canadian Open Neuroscience Platform is an open science infrastructure for sharing neuroscience data. More information can be found here.

Portal tutorials

We created a tutorial document on how to use the portal.

II. Data Access and Download

What data are contained in CONP?

CONP contains a diverse range of over 65 datasets (March 2022), primarily neuroimaging but also transcriptomics, genomics, and other related data modalities. Many of these datasets come from neuroscience research institutes, while others link to public resources that may be of interest to neuroscientists. A full list can be found here.

How do I download data from CONP?

Individual dataset pages accessible from the search page contain instructions for downloading data. At the moment, access to datasets is made available through either a one-click, browser-based download function or through the DataLad data management system, which should be version 0.12.2 or higher (installation details).

How do I access private datasets?

If a dataset requires a third-party account, please follow the URL indicated beside “Source” on the dataset page for further information.

III. Data Sharing/Upload

Why should I upload data to CONP?

The CONP Portal’s unique ability to federate data infrastructures through harmonized metadata provides unified search capabilities across normally siloed sources of datasets and analysis tools. "zThis is key to facilitating discoverability, reuse, and deeper scientific exploration of data, allowing, for instance, a researcher to discover multiple sources of data responding to given search criteria (e.g., according to modality, participant population, etc.), retrieve those data either through the Portal’s web interface or through command-line access that can be automated and integrated into an analysis pipeline, and apply containerized tools to these conjunctions of data either locally or through Cloud computing of various types. These features increase your data’s visibility, accessibility, and utility to the neuroscience community.

How do I upload data to CONP?

Data can be uploaded to CONP either through Zenodo by tagging your dataset with the keyword canadian-open-neuroscience-platform, through OSF by tagging your dataset with canadian-open-neuroscience-platform and setting the dataset to Public, or manually through DataLad. Detailed technical instructions are available for all these options.

IV. Tools & Pipelines

What tools/pipelines are contained in CONP?

CONP contains a diverse range of 80 tools and pipelines (March 2022). Many of these tools/pipelines come from neuroscience or genomics research institutes. A full list can be found here.

What are the requirements to install and run a tool?

CONP pipelines can be easily installed and run on any computer with a container engine. You will need to install:

Docker or Singularity
Boutiques python package (pip install boutiques)

How can I run a CONP tool/pipeline?

To run tools/pipelines locally, make sure your have Boutiques installed alongside either Docker or Singularity.

If you have never used Boutiques before, we recommend this tutorial.

Tools/pipelines can be run on Boutiques based on their Zenodo ID as follows:

The example command will create a first minimal invocation so that you don't have to start from scratch: bosh example <zenodo_id>. If you feel like starting with a more complete set of options, you can pass --complete to the example command: bosh example --complete <zenodo_id>
You can now edit a JSON example invocation (example_invocation.json) to add your input values based on the example provided in 1)
You are now all set to use the exec command to launch an analysis. Note, the option -s will show live stdout/stderr in the terminal.

bosh exec launch -s <zenodo_id> <invocation>

Note that in addition to the ability to run the tools/pipelines locally, it is possible to run some tools/pipelines via the CBRAIN platform (for example: the fmriprep BIDS App, Zenodo.3359640). Clicking on the blue icon "Use This Tool On CBRAIN" will bring you to the CBRAIN page where you can run pipelines on the super-computing network administered by the Digital Research Alliance of Canada. For more information on CBRAIN, visit http://www.cbrain.ca/.

I get the following error when running the tool on bosh

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
...
Exit code
125

=> ensure you have Docker installed and running before executing the bosh command.

</data/dataset> does not exist

Traceback (most recent call last):
  File "/run.py", line 53, in <module>
    run('bids-validator %s'%args.bids_dir)
  File "/run.py", line 25, in run
    raise Exception("Non zero return code: %d"%process.returncode)
Exception: Non zero return code: 2
...
Exit code
1

=> confirm your data directory is mounted on the container by using the -v option in the bosh command. For example, if your data is in /data, then you need to specify -v /data:/data in the bosh command as follows:bosh exec launch -s <zenodo_id> <invocation> -v <directory_to_mount>

How can I upload/share my own tool/pipeline on CONP?

You can add your own tool/pipeline by following the instructions described in this document.

V. DataLad

To which username/password is DataLad referring to when trying to download via `datalad get` preventad-open or multicenter-phantom datasets?

In order to have access to those datasets, you will need to request an account on their respective LORIS repositories.

To do so, on the dataset page, click on the link shown under "Source" which will redirect you to the LORIS login page. Below the login button there is a 'Request Account' link that will take you to a Request Account form that will need to be filled and submitted.

Once the account is approved, you will receive an email with the login credentials that you can use to either browse the LORIS repository or download data via DataLad using the datalad get command.

How to upgrade to DataLad version 0.12.2 or above?

In some environments, it is easier to install the latest version of DataLad via Miniconda by following the steps in the "Linux-machines with no root access (e.g. HPC systems)" sub-section of DataLad's Handbook Installation and Configuration page.

How can I update a dataset to benefit from the latest version?

From the conp-dataset root directory, run the following command to update all the datasets:

datalad update --merge