Validate an ELG compatible LT service (at technical/metadata level)

You can access the validation forms as described here.

When you are at the backend administration pages click on Technical validations to view the list of metadata records that have been assigned to you.

Catalogue for technical validators

Click on one of the metadata records to access the validation form:

Technical/Metadata validation form

Technical validation and service registration

On the validation form, the values of Technically valid and Metadata valid fields are set to Uknnown; do not change them yet.

Fill in the Slack channel and Jira ticket fields and click on “Save” at the right bottom corner of the page.

Before filling-in the rest of the validation form, you must first:

  • Deploy the LT service into the ELG kubernetes cluster: For this you have to create the required yaml file, as in the example below, in the respective GitLab repository and branch.

image : "registry.gitlab.com/qurator-platform/dfki/srv-ler:1.0.569798760"
limits_memory: 2048Mi
scalability: "dynamic"
minScale: "1"

To create the yaml file, you will need the docker_download_location, service_adapter_download_location and execution_location metadata element values, that you will find on the form. When the yaml file is commited/pushed, the automatic CI/CD deployment pipeline of ELG will be notified and install the LT service to the respective kubernetes cluster; for instance, the master branch is used for the production ELG cluster, while the develop branch is used for the development cluster. You will also need access to the cluster so that you can check which containers/pods are running, inpect the logs and statuses of the containers, etc.; for this, you will receive the required information and credentials from the ELG technical team.

  • Register/integrate the running LT service at the ELG catalogue: Click on the “LT Service Registration” button at the right top corner of the validation form.

The form will open at a new tab of your browser:

Service registration form

On this form:

  • select the Tool type value (IE, MT, ASR, etc.), depending on the type of service you validate. The metadata of the service will help you decide which is the appropriate value; you can view the metadata record by clicking on the “VIEW ON SITE” button on the top right corner of the validation form.

  • fill in the ELG execution location; the value of this field depends on how the LT service is deployed into the cluster; e.g. knative vs. non-knative (kubernetes) service, which namespace was used, etc. As described here, this field follows this template: http://{k8s service name for the registered LT tool}.{k8s namespace for the registered LT tool}.svc.cluster.local{the path where the REST service is running at}. The {the path where the REST service is running at} part can be filled based on the executionLocation element value of the metadata record. For this reason, the ELG execution location field id pre-filled with the executionLocation value, so that you can change only the part that is required.

  • set the Elg gui url; use

    • /dev/gui-ie/index-mt.html for MT services,

    • /dev/gui-udpipe/ for dependency parsers,

    • /dev/gui-ie/ for IE tools,

    • /dev/gui-tts/ for TTS services, and

    • /dev/gui-ie/index-asr.html for ASRs.

    To specify the text direction of the results in the try out UIs you should use the appropriate parameters; e.g. for ASRs use /dev/gui-ie/index-asr.html?dir=ltr or /dev/gui-ie/index-asr.html?dir=rtl for left-to-right and right-to-left directions, respectively. Similarly, /dev/gui-ie/?dir=ltr or /dev/gui-ie/?dir=rtl for IE services. For MT services use srcdir and targetdir parameters (e.g. index-mt.html?srcdir=ltr&targetdir=rtl) to specify the text direction of input/output. Depending on the value that you have set, the appropriate try out UI will be displayed in the respective tab of the landing page of the LT service. If there is no available/appropriate try out UI for the specific service or for any other reason you can disable/hide the try out UI tab by setting none to the Elg gui url field.

  • add the yaml file location in the respective field; it is the URL of the file that you have created in the previous step. This will help you easily locate in the future which yaml file specifies how the service is deployed into the cluster.

  • set the Accessor id of the service. This id is unique and it is used for calling the service via the ELG public LT REST API. For a brief overview of the LT REST API, see this section.

  • set the value of status to completed and click on “Save” at the bottom right corner of the page in order to activate the service

  • Τest the deployed service: You have to use the Try out tab of the service page (“VIEW ON SITE” will lead you there) or with any other available clients/tools. You must ensure that the service follows the ELG specifications and works as expected. For any issues that arise, you can use the specified slack channel to communicate with the provider of the service.

Now that the service is tested and integrated into ELG, you can return to the technical/metadata validation form and set Technically valid to Yes.

Then you have to check the metadata of the service.

Metadata validation

As already mentioned you can see the record by clicking the “VIEW ON SITE” button. You are asked to check whether the values of the following elements are included in the metadata record and whether their values match the description of the service:

  • function: important for findability purposes

  • input & output language(s): for MT services, the output language(s) must be included; for services of other types, the output language is not recommended (redundant information)

  • input & output data type(s): important for findability and interoperability purposes

  • output annotation type(s): if the tool is of the IE type, it’s recommended that this element has values for the types of information annotated/extracted

  • resource creator(s) and publication date: although not mandatory, they are useful for citation purposes;

  • domain(s): recommended for findability purposes; if possible, recommend the use of an existing value.

  • documentation: user and installation manuals for services are recommended; publications describing the use of the resource are also welcome

  • distribution(s): if a resource is available in multiple forms (e.g. as a functional service, but also as source code or downloadable form), it’s recommended to describe them as different distributions

  • software distribution form: check the values at https://european-language-grid.readthedocs.io/en/release1.1.1/Documentation/ELG-SHARE_xsd.html#SoftwareDistributionForm; depending on the form, a different element (access, download, execution or docker download location) is recommended.

If you are satisfied, set the values of Metadata valid to Yes and click on “Save”.

If not, add your comments and recommendations at the Review comments field and set the value of Metadata valid to No and click on “Save”, as below:

Technical validation form with review comments

The provider will be notified by email (containing the review comments) in order to update the record. Once finished, the provider will re-submit the record for publication and you will be notified to perform the validation again.

Please, keep in mind that an item is published only when it has been approved at all validation levels (technical, metadata and legal).

Note

This section is under development and continuously updated with new information. The forms included in the current release are implemented with Django. New forms will be provided later.