Guidelines for sharing NWB extensions (NDX)

• Version: 0.3.0 (DRAFT)
• Authors:
  • Oliver Ruebel
  • Andrew Tritt
  • Benjamin Dichter
  • Ryan Ly
• Last update: October 18, 2019

Overview

The purpose of this document is to define the requirements and strategy for sharing format extensions for NWB, so called Neurodata Extensions (NDX).

Definitions

• The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
• “Neurodata Extensions” (NDX) refer to extensions to the NWB data standard. NDX MUST be described by a formal format specification using the NWB specification language.
• “Internal NDX” refer to extensions used within a particular lab, organization, or project that are not intended for use outside of that group or context.
• “Public NDX” refer to extensions intended for use by the public (or specific larger community).
• “NDX Catalog” refers to the public catalog for NDX. The NDX Catalog is implemented via a dedicated public GitHub organization at https://github.com/nwb-extensions and website https://nwb-extensions.github.io/.
• “ndx-custom” refers to a specific user extension repository that has usually been created from the ndx-template and is used to manage the sources of the extension. This repository lives in the user’s own Git space and is NOT part of the nwb-extensions organization.
• “ndx-custom-record” refers to a specific record repository for deploying a particular extension as part of the NDX Catalog. This repo is part of the “nwb-extensions” GitHub organization. This repo is automatically generated by the CI of the staged-extensions repo and includes basic metadata about the NDX and information needed for install and deployment.

Requirements for sharing extensions

A Public NDX MUST follow the rules outlined below in order to be considered compliant. Internal NDX are also highly RECOMMENDED to follow the same rules outlined below as much as possible.

1. Releases of extensions MUST minimally include:
   1. The complete YAML source and namespace files for the NDX, which MUST be compliant with the NWB specification language. In addition to “name”, and “schema”, the namespace specification further MUST include the “author”, “contact”, and “version” of the namespace. The version number MUST follow the NWB semantic versioning rules.
   2. LICENSE file detailing the license for the NDX. Permissive licenses SHOULD be used if possible. BSD license is RECOMMENDED. In particular for release via the NDX Catalog (see Sec. 4ii), the license MUST include language giving permission to members of the GitHub organization to distribute the extensions.
   3. Release notes detailing the changes between extension versions. Release notes MAY be included in the detailed RST documentation or as a separate RELEASENOTES.md (or .rst) file.
   4. Requirements file detailing dependencies on other NDX and where they can be installed from. This is required as the namespace.yaml only includes the names of namespaces that are being included.
   5. README.md (or .rst) file with an overview of what the NDX does and additional details about the NDX.
2. Releases of extensions SHOULD further include:
   1. The sources for generating the YAML specification using the PyNWB/HDMF extensions APIs. Sharing the sources eases maintenance, testing, and reuse of the NDX.
   2. Additional documentation in Sphinx RST format describing the NDX, credits, legal, and acknowledgements.
   3. Examples and tutorials describing the use of the extension.
3. Releases of extensions MAY further include:
   1. Custom Python plugins (e.g., Container classes for neurodata_types) for PyNWB. This is RECOMMENDED to make the NDX accessible to end users and has the advantage that custom functionality for interacting with the data can be defined.
   2. Custom Matlab plugins for MatNWB for representing neurodata_types.
4. Performing releases:
   1. To ensure compliance with the semantic versioning rules, public NDX MUST be released in a persistent fashion (i.e., sources of a particular version MUST not be modified after release and MUST remain accessible).
   2. Public NDX MUST register with the NDX Catalog. To register with the catalog, a public NDX MUST create an ndx-custom-record Git repository as part of the nwb-extensions GitHub organization. An ndx-custom-record repo contains basic metadata about the NDX and information needed for install and deployment. The sources of the extension MAY be hosted in a different public ndx-custom repository.
   3. Public ndx-custom repositories SHOULD use Git tags to ensure compliance with (4i) and NDX SHOULD follow the NDX Catalog’s sharing strategies. Note: software versions managed via Git tags MAY differ from the version of the extensions namespace. However, both MUST follow NWB semantic versioning rules.
5. Naming namespaces and repositories:
   1. The name key of the namespace for the extension SHOULD generally follow the following naming conventions:
      1. Start with the prefix “ndx-”, e.g., “ndx-cortical-surface” (required for registration with the NDX Catalog)
      2. Use short and descriptive names
      3. Use only lower-case ASCII letters (no special characters)
      4. Use “-” to separate different parts of the name (no spaces allowed)
   2. The namespace YAML file with the specification of the namespace SHOULD have the same name as the name key of the main namespace and the namespace.yaml postfix SHOULD be added. E.g., if the name of the namespace is “ndx-cortical-surface”, then the YAML file would be called “ndx-cortical-surface.namespace.yaml”.
   3. The ndx-custom repository of an extension SHOULD generally follow the following conventions:
      1. Have the same name as the name key of the main namespace of the NDX (i.e., “ndx_myname”)(e.g., “ndx-cortical-surface”)
      2. Use GitHub (preferred) or other public Git hosting service
      3. Include a description
      4. Specify at least the following GitHub topics: ndx-extension
   4. The Python package for the NDX SHOULD have the same name as the name key of the main namespace of the NDX, where hyphens are replaced with underscores (i.e., “ndx_myname”)(e.g., from ndx_cortical_surface import CorticalSurface).
   5. The ndx-custom-record repository as part of the NDX Catalog MUST have the same name as the name key of the main namespace of the NDX i.e., “ndx-myname”)(e.g., “ndx-cortical-surface”). This strategy helps ensure that there are no name conflicts between public NDX namespace names. This repository is created automatically, so end users need not worry about this.