Validate an ELG compatible LT service (at technical/metadata level)

See here how to access the “Validation Tasks”, which is the list of items assigned to you for validation. You can, then, apply the filters on the left to help you reduce the number of items presented or search for a specific item using the search box.

Technical Validations Tasks

Deployment of the service at ELG and service registration

Before you perform the technical/metadata validation, you must first deploy the service at ELG and register it at the ELG catalogue.

Deploy the LT service into the ELG kubernetes cluster: For this you have to create the required yaml file, as in the example below, in the respective GitLab repository and branch.

image : "registry.gitlab.com/qurator-platform/dfki/srv-ler:1.0.569798760"
limits_memory: 2048Mi
scalability: "dynamic"
minScale: "1"

To create the yaml file, you will need the docker_download_location, service_adapter_download_location and execution_location metadata element values, that you will find on the service registration form. By selecting one of the metadata records on the “My validations” tab, you will be directed to the its view page; click on the actions box and select “Perform service registration”.

Technical/Metadata validation form

When the yaml file is commited/pushed, the automatic CI/CD deployment pipeline of ELG will be notified and install the LT service to the respective kubernetes cluster; for instance, the master branch is used for the production ELG cluster, while the develop branch is used for the development cluster. You will also need access to the cluster so that you can check which containers/pods are running, inpect the logs and statuses of the containers, etc.; for this, you will receive the required information and credentials from the ELG technical team.

On the service registration form, you must also provide the following necessary information

Service registration form
  • select the Tool type value (IE, MT, ASR, etc.), depending on the type of service you validate. The metadata of the service will help you decide which is the appropriate value;

  • fill in the ELG execution location; the value of this field depends on how the LT service is deployed into the cluster; e.g. knative vs. non-knative (kubernetes) service, which namespace was used, etc. As described here, this field follows this template: http://{k8s service name for the registered LT tool}.{k8s namespace for the registered LT tool}.svc.cluster.local{the path where the REST service is running at}. The {the path where the REST service is running at} part can be filled based on the executionLocation element value of the metadata record. For this reason, the ELG execution location field id pre-filled with the executionLocation value, so that you can change only the part that is required.

  • set the Elg gui url; use

    • /dev/gui-ie/index-mt.html for MT services,

    • /dev/gui-udpipe/ for dependency parsers,

    • /dev/gui-ie/ for IE tools,

    • /dev/gui-tts/ for TTS services, and

    • /dev/gui-ie/index-asr.html for ASRs.

    To specify the text direction of the results in the try out UIs you should use the appropriate parameters; e.g. for ASRs use /dev/gui-ie/index-asr.html?dir=ltr or /dev/gui-ie/index-asr.html?dir=rtl for left-to-right and right-to-left directions, respectively. Similarly, /dev/gui-ie/?dir=ltr or /dev/gui-ie/?dir=rtl for IE services. For MT services use srcdir and targetdir parameters (e.g. index-mt.html?srcdir=ltr&targetdir=rtl) to specify the text direction of input/output. Depending on the value that you have set, the appropriate try out UI will be displayed in the respective tab of the landing page of the LT service. If there is no available/appropriate try out UI for the specific service or for any other reason you can disable/hide the try out UI tab by setting ``none`` to the ``Elg gui url`` field.

  • set the Accessor id of the service. This id is unique and it is used for calling the service via the ELG public LT REST API. For a brief overview of the LT REST API, see this section.

  • the execution location is pre-filled with the respective metadata value;

  • the docker download location is pre-filled with the respective metadata value;

  • set the value of status to completed and click on “Submit” in order to activate the service.

Τest the deployed service: You have to use the Try out tab of the service view page or with any other available clients/tools. You must ensure that the service follows the ELG specifications and works as expected. For any issues that arise, you can use the specified slack channel to communicate with the provider of the service.

Now that the service is tested and integrated into ELG, you can continue with the technical/metadata validation.

Technical/Metadata validation

Go to the item view page and click on the actions box to proceed.

Technical/Metadata validation form

A form will open in which you must say whether you Approve or Reject the item after the technical and metadata validation. You must provide both answers before you submit the form.

For the Metadata validation you are asked to check whether the values of the following elements are included in the metadata record and whether their values match the description of the service:

  • function: important for findability purposes

  • input & output language(s): for MT services, the output language(s) must be included; for services of other types, the output language is not recommended (redundant information)

  • input & output data type(s): important for findability and interoperability purposes

  • output annotation type(s): if the tool is of the IE type, it’s recommended that this element has values for the types of information annotated/extracted

  • resource creator(s) and publication date: although not mandatory, they are useful for citation purposes;

  • domain(s): recommended for findability purposes; if possible, recommend the use of an existing value.

  • documentation: user and installation manuals for services are recommended; publications describing the use of the resource are also welcome

  • distribution(s): if a resource is available in multiple forms (e.g. as a functional service, but also as source code or downloadable form), it’s recommended to describe them as different distributions

  • software distribution form: check the values at https://european-language-grid.readthedocs.io/en/stable/Documentation/ELG-SHAREschema.html#SoftwareDistributionForm; depending on the form, a different element (access, download, execution or docker download location) is recommended.

If you are satisfied, approve both types of validation and click on “Submit”.

If not, set the value of the Technical validation and/or the Metadata validation (depending on the source of the issue) to Reject. This will generate a new field where you can write the recommendations you would like to share with the curator. You can also add comments in the Validator notes field which will be visible only to other validators. When you have finished, click on submit.

Technical/metadata validation form with review comments

The provider will be notified by email (containing the review comments) in order to update the record. Once finished, the provider will re-submit the record for publication and you will be notified to perform the validation again.

Please, keep in mind that an item is published only when it has been approved at all validation levels (technical, metadata and legal).