Język: Polski | English
In case of problems: click here

Translator i Słownik Polskiego Języka Migowego

About the THETOS system

Project done at the Silesian University of Technology, financed by the Polish Committee for Scientific Research and the Ministry of Education and Science under research grants no 8 T11C 007 17 and 4 T11C 024 24. Copyright (c) 1999-2010 by the Silesian University of Technology.


The THETOS project is aimed at developing a system for translation of Polish texts into the Polish Sign Language (or – more precisely – into some PSL dialect). The translated sign utterance is shown by a virtual, animated interpreter.
At present the following elements are available:

  • a working prototype of the system (for internal use only),
  • a simplified demo version

The new version of the THETOS system is also available through a Web browser as a Unity3D applet. The adopted technology allows for translation of any complex sentences and texts. Words unknown to the system are spelled using the finger alphabet. All computations are done in the real time, thanks to that the system is fully interactive. On this site we introduce an applet, which in real-time demonstrates sign language signs that correspond to given words and word sequences. In this demo version only a small set of words is available. In the same time there is a more exhaustive (but incomplete) list of words, whose corresponding signs appear in the full version.

 

Goal and motivation of the project

Problems of hearing-impaired people are very often underestimated, as they are generally thought to be able to freely use text. However it often happens that the deaf-born people cannot neither write nor read or have serious problems with doing this. That is why using the sign language, which is their basic and natural means of communication, is so important. The main goals that we have set in the THETOS project are:

  • to help the communication between the hearing and non-hearing people,
  • to help the not hearing people to learn spoken (phonic) language,
  • to help the instruction of Polish Sign Language teachers and interpreters (both hearing and non-hearing).

The THETOS system allows for typing in a sentence (or a set of sentences) and then translates it to a sign language utterance. The solution respects a specific syntax and grammar of the Polish Sign Language which is remarkably different than that for the phonic language. The utterance, reduced to a sequence of gestures, in the final translation phase is shown by an animated, virtual character.
Additionally, the system has been supplemented by an animated dictionary of the PSL.



How the THETOS system is constructed and how it works

The THETOS system is composed of two main parts: the linguistic and the multimedia one.


The linguistic part transforms the input text into a sequence of words, which represent signs of the equivalent output utterance, built according to the sign language rules. Input utterance is passed through morphologic, syntactic and semantic analysis. In the course of the latter, selected anaphoric connections that exist in the text are detected and handled. The analyzed utterance is put in a predicate-argument representation, which works for a semantic representation. The transfer (proper translation) is done on this representation level. A limited but larger than in the prototypic version number of semantic relations is used. Linguistic processors with a block of dictionaries used by the linguistic part to perform analysis and translation, are gathered on the LAS sever. The server is accessible for all interested Internet users at address http://las.aei.polsl.pl/las2/.


In the multimedia part of the system, the words of the output utterance are demonstrated in form of signs-gestures by an intentionally elaborated virtual character – an avatar. Till 2006, the multimedia part was developed on the basis of the FreeWill+ platform. In 2007 we decided to switch to the FRS platform. In the FreeWill+ version, the movements composing to gestures were described using the B. Szczepankowski Gestographic Notation, SiGN. Procedures used to interpret gesture specifications – among other things –were supplementing implicit information about placement of the character’s body parts placement as well as eliminating specification inconsistencies. Most of body part collisions were eliminated. An additional implemented element was animation of movements of lips uttering words corresponding to signs being just demonstrated. The presentation is smooth and proceeds in the real-time. In the FRS version we resigned of the SiGN notation because of difficulties with obtaining on this basis a satisfactory movement precision. Instead the animation mechanism makes use of encoded detailed avatar movements description, registered during a session of direct modeling of those movements by a human. Lips movement animation has not been implemented yet.


The scope of texts being translated is limited with the content of dictionaries built for the input and the output languages. On the input side, the dictionaries of the THETOS 3.0 system contain description of about 75,000 words reduced to the basic form. On the output side, the dictionary for the version using the FreeWill+ platform contain ca. 1400 PSL signs, in this count all that are required from candidates for sign language interpreters during the first-level examination. A great deal of signs refers to the problems of first medical aid, for which the prototypic version of the system was intended. As to the version using the FRS platform, it contains some 1000 signs.



THETOS in media

The THETOS system has attired interest of media, in that of TV (regional and all-Poland’s) and press. We have been mentioned among other things:

  • in all-Poland’s educational emission (program edukacyjny) "Laboratorium" ("Laboratory") (TVP1),
  • in all-Polands news emission (program informacyjny) Panorama (TVP2),
  • in regional news emission (Aktualności - TVP3),
  • in popular science emission on the Polish Radio Bis,
  • in the report on the Polish Radio Science Picnic (30.05.2009) (PR3),
  • in the local supplement to "Gazeta Wyborcza",
  • in the local „Dziennik Zachodni” newspaper,
  • in some Internet services (among other things in Onet).

In the near future we will try to provide more stuff to this topic.



Stuff for download

The program is in the stage of continuous development. Now we are working on fine-tuning of the animated character movements. Not all gestures resemble the actual signs of the sign language and not all of them look natural. The stuff, which can be downloaded below, gives a good view of the current state of works.

Animation

Sample sign utterance generated by means of the THETOS system (the "old" FreWill+ platform).

Demo version

Version built using the "old" FreeWill+ platform. It allows for replaying a predefined set of sentences. Full linguistic analysis is blocked, the set of dictionary entries is reduced.

Download demo

Publications

To have a holistic view of problems related to the THETOS project please read the following publication:


J. Romaniuk, N. Suszczańska (2009)
Studies on Emotions Translation in the Thetos System.
In: GESPIN Conference, Poznań, Poland, CD ISBN 978-83-926446-7-5.

N. Suszczańska, P. Szmal, K. Simiński (2009)
The Deep Parser for Polish. In: Z. Vetulani, H. Uszkoreit,
"Responding to Information Society Challenges: New Advances in Human Language Technologies". LNAI 5603, Springer, pp. 205-211.

N. Suszczańska (2008)
System LIANA wspomagania projektowania systemów informatycznych.,
In: B. Hnatkowska, Z. Huzar (eds.) "Inżynieria oprogramowania – metody i narzędzia wytwarzania oprogramowania", Wydawnictwo Naukowe PWN, Warszawa, pp. 146-159.



See the full publication list

THETOS online

Enter word to translate:

You can also choose a word from the list: