Using the ELG services¶
Run an ELG service directly from Python.
[1]:
from elg import Service
Authentication¶
To use the ELG services, you need to have an ELG account and authenticate. If you are not registered in ELG yet, you can do it directly in the ELG website.
Once your account is setup, you need to authenticate inside the elg
library, and to do so, you need to obtain the access_tokens
. There are two ways to do it.
Directly authenticate in the Service
class¶
The easiest way to start using ELG services is simply to init a service based on its id. The initialization will take care of the authentication and ask you to connect in the ELG website. The access_tokens
will be obtained automaticly and saved in cache for further use.
[2]:
lt = Service.from_id(474)
The service is then ready to be used.
[3]:
result = lt("Nikolas Tesla lives in Berlin.")
print(f"\n{result}")
Calling:
[474] Cogito Discover Named Entity Recognizer
with request:
type: text - content: Nikolas Tesla lives in Berlin. - mimeType: text/plain
type='annotations' warnings=None features=None annotations={'People': [Annotation(start=0, end=13, source_start=None, source_end=None, features={'SURNAME': 'Tesla', 'SEX': 'M', 'name': 'Nikolas Tesla', 'NAME': 'Nikolas'})], 'Place': [Annotation(start=23, end=29, source_start=None, source_end=None, features={'Lemma': 'Berlin', 'name': 'Berlin', 'Glossa': 'Staatshauptstadt in Berlin (Deutschland/Europa', 'GEOREF': 'Berlin/Deutschland/Europa'})]}
As the tokens are now stored in cache, you can init another service without having to login again.
[4]:
lt = Service.from_id(7289)
The Service
class is using the Authentification
class to deal with the access_tokens
. You can access the tokens via the authentication
attribute. It can be useful to save the tokens in a different json
file or print the validation deadline of the tokens.
[5]:
lt.authentication.to_json("another_tokens.json")
print(f"The tokens will expire the {lt.authentication.refresh_expires_time}")
The tokens will expire the time.struct_time(tm_year=2021, tm_mon=11, tm_mday=8, tm_hour=20, tm_min=53, tm_sec=18, tm_wday=0, tm_yday=312, tm_isdst=0)
As you can see, the obtained tokens will expire soon and will work for only a couple of hours.
To obtain offline tokens that won’t expire, you need to change the scope
parameters of the Service
initialization to "offline_access"
. As the tokens asked are different from the ones saved in cache, you will need to authenticate again.
[6]:
lt = Service.from_id(474, scope="offline_access")
print(f"The tokens will expire the {lt.authentication.refresh_expires_time}")
The tokens will expire the time.struct_time(tm_year=2121, tm_mon=10, tm_mday=15, tm_hour=16, tm_min=0, tm_sec=30, tm_wday=2, tm_yday=288, tm_isdst=0)
Use the Authentication
class and use the tokens in the Service
class¶
The authentication is the main component to obtain the access_tokens
. You have to init()
the authentication and then save the obtained tokens into a json
file to use to init a service.
[7]:
from elg import Authentication
auth = Authentication.init(scope="openid")
auth.to_json("tokens.json")
print(f"\n\nThe tokens will expire the {auth.refresh_expires_time}")
Please go to this URL in your browser: https://live.european-language-grid.eu/auth/realms/ELG/protocol/openid-connect/auth?client_id=elg-oob&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=openid
Paste the "success code": 41ebafbf-89fd-4f5f-bc82-3353cf58297a.34c84bea-64c8-40d7-ae1b-2f5178aafbb1.7f70e03d-f327-4333-8ec9-2b236b432169
The tokens will expire the time.struct_time(tm_year=2021, tm_mon=11, tm_mday=8, tm_hour=20, tm_min=53, tm_sec=18, tm_wday=0, tm_yday=312, tm_isdst=0)
As previously, you can change the scope
parameter to obtain offline tokens.
[8]:
auth = Authentication.init(scope="offline_access")
auth.to_json("tokens.json")
print(f"\n\nThe tokens will expire the {auth.refresh_expires_time}")
Please go to this URL in your browser: https://live.european-language-grid.eu/auth/realms/ELG/protocol/openid-connect/auth?client_id=elg-oob&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=offline_access
Paste the "success code": 4db695af-1dd0-414e-80ce-12f0f903e9ba.34c84bea-64c8-40d7-ae1b-2f5178aafbb1.7f70e03d-f327-4333-8ec9-2b236b432169
The tokens will expire the time.struct_time(tm_year=2121, tm_mon=10, tm_mday=15, tm_hour=16, tm_min=10, tm_sec=33, tm_wday=2, tm_yday=288, tm_isdst=0)
The obtained tokens saved in the json
file can then be used to use the services without login again.
[9]:
lt = Service.from_id(474, auth_file="tokens.json")
Using authentication file: tokens.json
Initialization¶
You can initialize a service from its id or you can also init a service from an Entity
.
Using from_id
¶
[10]:
lt = Service.from_id(474)
Using from_entity
¶
We can use the catalog to search a Named Entity Recognizer for French and init a Service with the returned Entity.
[11]:
from elg import Catalog
catalog = Catalog()
results = catalog.search(
resource = "Tool/Service",
function = "Named Entity Recognition",
languages = ["fr"],
limit = 1,
)
entity = next(results)
print(entity)
lt = Service.from_entity(entity=entity)
result = lt("Jean Dupond vit à Paris.")
print(f"\n{result}")
----------------------------------------------------------------------
Id 474
Name Cogito Discover Named Entity Recognizer
Resource type Tool/Service
Entity type LanguageResource
Description Annotation of entities: People, Organizations, Places,
Known concepts, Unknown concepts. And also tags: urls,
mail addresses, phone numbers, addresses, dates, time,
measures, money, percentage, file folder.
Licences ['Cogito Discover License']
Languages ['French', 'Dutch', 'Portuguese', 'English', 'German',
'Italian', 'Spanish']
Status None
----------------------------------------------------------------------
Calling:
[474] Cogito Discover Named Entity Recognizer
with request:
type: text - content: Jean Dupond vit à Paris. - mimeType: text/plain
type='annotations' warnings=None features=None annotations={'People': [Annotation(start=0, end=11, source_start=None, source_end=None, features={'SURNAME': 'Dupond', 'SEX': 'M', 'name': 'Jean Dupond', 'NAME': 'Jean'})], 'Place': [Annotation(start=18, end=23, source_start=None, source_end=None, features={'Lemma': 'Paris', 'name': 'Paris', 'Glossa': 'capitale in Paris (Île-de-France/France/Europe', 'GEOREF': 'Paris/Île-de-France/France/Europe'})]}
Usage¶
To call the services, you can use either a plain text as before, a file, or a Request
object.
Call the service using plain text¶
[12]:
lt = Service.from_id(474)
result = lt("Nikolas Tesla lives in Berlin.")
print(f"\n{result}")
Calling:
[474] Cogito Discover Named Entity Recognizer
with request:
type: text - content: Nikolas Tesla lives in Berlin. - mimeType: text/plain
type='annotations' warnings=None features=None annotations={'People': [Annotation(start=0, end=13, source_start=None, source_end=None, features={'SURNAME': 'Tesla', 'SEX': 'M', 'name': 'Nikolas Tesla', 'NAME': 'Nikolas'})], 'Place': [Annotation(start=23, end=29, source_start=None, source_end=None, features={'Lemma': 'Berlin', 'name': 'Berlin', 'Glossa': 'Staatshauptstadt in Berlin (Deutschland/Europa', 'GEOREF': 'Berlin/Deutschland/Europa'})]}
Call the service using a file¶
You can create a simple text file that contains the input you want to use to call the service.
[13]:
!rm example.txt && echo "Jean Dupond vit à Paris." >> example.txt
You can pass the path to the file you just created. For audio file, you can do the same, except you need to set the request_type
parameter to audio
.
[14]:
result = lt("example.txt")
print(f"\n{result}")
Calling:
[474] Cogito Discover Named Entity Recognizer
with request:
type: text - content: Jean Dupond vit à Paris.
- mimeType: text/plain
type='annotations' warnings=None features=None annotations={'People': [Annotation(start=0, end=11, source_start=None, source_end=None, features={'SURNAME': 'Dupond', 'SEX': 'M', 'name': 'Jean Dupond', 'NAME': 'Jean'})], 'Place': [Annotation(start=18, end=23, source_start=None, source_end=None, features={'Lemma': 'Paris', 'name': 'Paris', 'Glossa': 'capitale in Paris (Île-de-France/France/Europe', 'GEOREF': 'Paris/Île-de-France/France/Europe'})]}
Call the service using a Request
object¶
The SDK contains a representation of each ELG request message that can be used as input when calling a service.
[15]:
from elg.model import TextRequest
request = TextRequest(content="Jean Dupond vit à Paris.")
result = lt(request)
print(f"\n{result}")
Calling:
[474] Cogito Discover Named Entity Recognizer
with request:
type: text - content: Jean Dupond vit à Paris. - mimeType: text/plain
type='annotations' warnings=None features=None annotations={'People': [Annotation(start=0, end=11, source_start=None, source_end=None, features={'SURNAME': 'Dupond', 'SEX': 'M', 'name': 'Jean Dupond', 'NAME': 'Jean'})], 'Place': [Annotation(start=18, end=23, source_start=None, source_end=None, features={'Lemma': 'Paris', 'name': 'Paris', 'Glossa': 'capitale in Paris (Île-de-France/France/Europe', 'GEOREF': 'Paris/Île-de-France/France/Europe'})]}
Advanced usages¶
You can apply a method to the result to extract the information needed. To do so, you have to pass a callable object in the ouput_func
parameter.
[16]:
service = Service.from_id(5228)
pretty_result = service("Ich habe diesen Film geliebt. Die Schauspieler, das Drehbuch: alles von einem Meisterwerk.", output_func=lambda x: x.texts[0].content)
print("Translation to Finnish: ", pretty_result)
Calling:
[5228] OPUS-MT: German-Finnish machine translation
with request:
type: text - content: Ich habe diesen Film geliebt. Die Schauspieler, das Drehbuch: alles von einem Meisterwerk. - mimeType: text/plain
Translation to Finnish: Rakastin tätä elokuvaa. Näyttelijät, käsikirjoitus: Kaikki mestariteoksesta.
You can also set the output_func
parameter to “auto” to extract the information needed automaticly. This is not working for all the services.
[17]:
service = Service.from_id(5228)
pretty_result = service("Ich habe diesen Film geliebt. Die Schauspieler, das Drehbuch: alles von einem Meisterwerk.", output_func="auto")
print("Translation to Finnish: ", pretty_result)
Calling:
[5228] OPUS-MT: German-Finnish machine translation
with request:
type: text - content: Ich habe diesen Film geliebt. Die Schauspieler, das Drehbuch: alles von einem Meisterwerk. - mimeType: text/plain
Translation to Finnish: Rakastin tätä elokuvaa. Näyttelijät, käsikirjoitus: Kaikki mestariteoksesta.