das : RSS Feed

minutes260508

05/27/08 20:30

- InPro, meeting, minutes, 26/05/08

- present: T, M, D

- acoustic model: improving. coming along.

- still to do:

- from microphone to asr to parser

- n-best, confidences

- parser:

- go parallel approach? Have both parser and pattern matcher that

simply looks at the string so far and tries to grab from it

whatever kinds of expressions it knows. Sort of named entity

recognition..

.. but how can this be done incrementally?

- discussed abstr incr model:

- problem w/ meaning of grounded_in vs same-level-link. Is not an

S node grounded in its constituents, rather than the string?

Design decision, ultimately. All we want is that the necessary

information is transmitted. E.g., if a VP which is a constituent

of an S is withdrawn (parser removes grounded-in, because it

doesn't think anymore that the evidence (the words) justify

hypothesis of VP), the S must be marked as ungrounded as

well. This can be given to the processor as a housekeeping task

(if it ungrounds stg, it must also unground everything it has

built on it), it can be added to the purge function (if an LB-IU

has become ungrounded, purge it and everything that links to it

on same level), or it can be done automatically through

interleaving of grounded-in and same-level-link (the grounded-in

of larger constituents can be expressed through reference to the

grounded-in of its same-level constituents, in which case every

change in their status automatically percolates up). Last option

is probably the most elegant.

- parser as semi-circular module. Constituents it has built are

put on LB, to be consumed together with LB-IUs coming from

previous module.

- question of boundaries of task. Is this also used to model

dialogue history? The links btw utterances? Could be done, but

probably practically not useful.. But if done, maybe useful to

have commit as 3-way distinction. `freeze', but allow to go back

later and revise it, e.g. after a misunderstanding. (?).

- does the model have to say something about how to avoid timing

issues? Race conditions, starving, etc. Probably not. If time,

one could think about how problematic conditions could arrive

(e.g., a purge coming to late, after a hypothesis has been

extended, thus failing to percolate quickly enough through the

network. -- Although this probably could be avoided if the

processor does its functions in sequence each time its triggered

into action, with purge coming first.)

minutes260508

05/27/08 20:30

- InPro, meeting, minutes, 26/05/08

- present: T, M, D

- acoustic model: improving. coming along.

- still to do:

- from microphone to asr to parser

- n-best, confidences

- parser:

- go parallel approach? Have both parser and pattern matcher that

simply looks at the string so far and tries to grab from it

whatever kinds of expressions it knows. Sort of named entity

recognition..

.. but how can this be done incrementally?

- discussed abstr incr model:

- problem w/ meaning of grounded_in vs same-level-link. Is not an

S node grounded in its constituents, rather than the string?

Design decision, ultimately. All we want is that the necessary

information is transmitted. E.g., if a VP which is a constituent

of an S is withdrawn (parser removes grounded-in, because it

doesn't think anymore that the evidence (the words) justify

hypothesis of VP), the S must be marked as ungrounded as

well. This can be given to the processor as a housekeeping task

(if it ungrounds stg, it must also unground everything it has

built on it), it can be added to the purge function (if an LB-IU

has become ungrounded, purge it and everything that links to it

on same level), or it can be done automatically through

interleaving of grounded-in and same-level-link (the grounded-in

of larger constituents can be expressed through reference to the

grounded-in of its same-level constituents, in which case every

change in their status automatically percolates up). Last option

is probably the most elegant.

- parser as semi-circular module. Constituents it has built are

put on LB, to be consumed together with LB-IUs coming from

previous module.

- question of boundaries of task. Is this also used to model

dialogue history? The links btw utterances? Could be done, but

probably practically not useful.. But if done, maybe useful to

have commit as 3-way distinction. `freeze', but allow to go back

later and revise it, e.g. after a misunderstanding. (?).

- does the model have to say something about how to avoid timing

issues? Race conditions, starving, etc. Probably not. If time,

one could think about how problematic conditions could arrive

(e.g., a purge coming to late, after a hypothesis has been

extended, thus failing to percolate quickly enough through the

network. -- Although this probably could be avoided if the

processor does its functions in sequence each time its triggered

into action, with purge coming first.)

Home Page

05/27/08 20:27

Besprechungsprotokolle / meeting minutes

(newest first)

26/05/08 minutes260508

19/05/08 minutes190508

05/05/08 minutes080508

14/04/08 minutes140408

03/02/08 minutes030208b

04/12/07 minutes041207

26/11/07 @Timo

19/11/07 minutes191107

13/11/07 minutes131107

05/11/07 minutes051107

22/10/07 minutes221007

01/10/07 minutes2007_10_01

10/09/07 minutes100907

23/08/07 minutes230807

03/07/07 minutes030707

19/06/07 minutes190607

05/06/07 minutes050607_zeitwort2

21/05/07 minutes210507

Sonstiges

Conferences2008

minutes190508

05/20/08 10:01

- present: M, T, D

1. akustisches Modell

- momentan nicht besonders gut. Trainiert mit Daten von vox forge

(?).

- to be redone with Pento Naming Corpus data

- also add Verbmobil data? Kiel Korpus?

- when that is done, tackle other points:

- LexTree instead of SimpleLM, so that tri-gram LMs can be used

- n-best lists, or lattices

- confidence scores

--> by end of May, we will have model that is good enough to at

least get an idea of what we will be working with.

Germany's next top (acoustic) model!

2. literature day(s)

- topics:

- parsing / semantics for SDS

- incremental parsing (in general)

- parsing and prosody

- incremental systems

- EOT prediction / recognition

----> collect paper suggestion on Wiki. Michaela will organise

the (first) day.

X. brief interlude: do we need new collaboration management system?

What do we use at the moment:

- email .. anouncements, rarely for content, moving of files

- pro:

- archivable / searchable

- attachments

- IM .. quick questions (T and D; M doesn't use at all)

- pro:

- fast, instant (d'uh)

- cons:

- not easy to search. breaks unity of transmission (where did

I read that?)

- Elgg .. mostly for meeting minutes (and there the wiki part is

used only). T uses it for status reports. M doesn't

use it at all. Not used for literature notes etc.

- pro:

- archival.. ?

- cons:

- active effort needed to put things there & to check for new

entries (since integration of RSS in our workflows doesn't

seem to work yet)

What we'd need:

- ideal would be a system that has more than one interface,

including email. That is, new content can be contributed by

email, web, whatever, is spread via email, can be searched in

one central place.

Bonus: has interface to svn, e.g. one can link to documents in

the svn. also has IM client, and archives IMs.

.. is that trac? probably not. Does that exist? Probably not.

3. WOz

- controls mouse and prompts

- data can be used for acoustic model and language model, also

for learning about dialogue dynamics that can be expected. Main

goal is to see how people behave if they assume that the

capabilities of the instruction follower are limited.

4. Parser, requirements

- robustness. Can't assume that it always will be able to parse

into sentences. Should be happy with intermediate

constituents.

Doesn't this requirement fall out of incrementality in any

case? If partial results are passed on, they always will be

sub-S constiuents.

Yes, but the problem has a slightly different aspect as

well. The question is what to do if what the parser can

possibly recognise (because of ASR problems) are sequences of

NPs. If there is no syntactic rule that could potentially

integrate those, if not specially prepared, the parser would

not even attempt to build the later NPs.

So what it boils down to again is the question of whether the

parser needs a notion of being "restarted" or not.

- incrementality. d'uh.

- non-commital, capable of making revisions

related to topic above. What happens if the parser decides to

give up integrating new material into the current structure?

- mid-term: probabilistic, integrate prosody (as information on

words or as pseudo tokens), parse lattices

Discussed:

- is top-down parsing a good idea for an incremental parser? can

this work? Think through, what are the problems that can arise

in either case?

--> Michaela? Think up a few example cases (including ambiguous

and garbage-full sentences) and see what either parsing

strategy will do.

- how is commital (when parser decides that what it currently is

consuming is unrelated to structure it has previously built)

realised technically, how is it triggered?

Cleaning datastructures. Empty chart, or un-link datastructure?

minutes190508

05/20/08 10:01

- present: M, T, D

1. akustisches Modell

- momentan nicht besonders gut. Trainiert mit Daten von vox forge

(?).

- to be redone with Pento Naming Corpus data

- also add Verbmobil data? Kiel Korpus?

- when that is done, tackle other points:

- LexTree instead of SimpleLM, so that tri-gram LMs can be used

- n-best lists, or lattices

- confidence scores

--> by end of May, we will have model that is good enough to at

least get an idea of what we will be working with.

Germany's next top (acoustic) model!

2. literature day(s)

- topics:

- parsing / semantics for SDS

- incremental parsing (in general)

- parsing and prosody

- incremental systems

- EOT prediction / recognition

----> collect paper suggestion on Wiki. Michaela will organise

the (first) day.

X. brief interlude: do we need new collaboration management system?

What do we use at the moment:

- email .. anouncements, rarely for content, moving of files

- pro:

- archivable / searchable

- attachments

- IM .. quick questions (T and D; M doesn't use at all)

- pro:

- fast, instant (d'uh)

- cons:

- not easy to search. breaks unity of transmission (where did

I read that?)

- Elgg .. mostly for meeting minutes (and there the wiki part is

used only). T uses it for status reports. M doesn't

use it at all. Not used for literature notes etc.

- pro:

- archival.. ?

- cons:

- active effort needed to put things there & to check for new

entries (since integration of RSS in our workflows doesn't

seem to work yet)

What we'd need:

- ideal would be a system that has more than one interface,

including email. That is, new content can be contributed by

email, web, whatever, is spread via email, can be searched in

one central place.

Bonus: has interface to svn, e.g. one can link to documents in

the svn. also has IM client, and archives IMs.

.. is that trac? probably not. Does that exist? Probably not.

3. WOz

- controls mouse and prompts

- data can be used for acoustic model and language model, also

for learning about dialogue dynamics that can be expected. Main

goal is to see how people behave if they assume that the

capabilities of the instruction follower are limited.

4. Parser, requirements

- robustness. Can't assume that it always will be able to parse

into sentences. Should be happy with intermediate

constituents.

Doesn't this requirement fall out of incrementality in any

case? If partial results are passed on, they always will be

sub-S constiuents.

Yes, but the problem has a slightly different aspect as

well. The question is what to do if what the parser can

possibly recognise (because of ASR problems) are sequences of

NPs. If there is no syntactic rule that could potentially

integrate those, if not specially prepared, the parser would

not even attempt to build the later NPs.

So what it boils down to again is the question of whether the

parser needs a notion of being "restarted" or not.

- incrementality. d'uh.

- non-commital, capable of making revisions

related to topic above. What happens if the parser decides to

give up integrating new material into the current structure?

- mid-term: probabilistic, integrate prosody (as information on

words or as pseudo tokens), parse lattices

Discussed:

- is top-down parsing a good idea for an incremental parser? can

this work? Think through, what are the problems that can arise

in either case?

--> Michaela? Think up a few example cases (including ambiguous

and garbage-full sentences) and see what either parsing

strategy will do.

- how is commital (when parser decides that what it currently is

consuming is unrelated to structure it has previously built)

realised technically, how is it triggered?

Cleaning datastructures. Empty chart, or un-link datastructure?

Home Page

05/20/08 10:00

Besprechungsprotokolle / meeting minutes

(newest first)

19/05/08 minutes190508

05/05/08 minutes080508

14/04/08 minutes140408

03/02/08 minutes030208b

04/12/07 minutes041207

26/11/07 @Timo

19/11/07 minutes191107

13/11/07 minutes131107

05/11/07 minutes051107

22/10/07 minutes221007

01/10/07 minutes2007_10_01

10/09/07 minutes100907

23/08/07 minutes230807

03/07/07 minutes030707

19/06/07 minutes190607

05/06/07 minutes050607_zeitwort2

21/05/07 minutes210507

Sonstiges

Conferences2008

minutes080508

05/08/08 10:22

- present: M, T, D

- priority: close processing chain. Finally get something from ASR

to parser to DM to TTS -- even if it is only a parrot system!

- Dialogue Manager:

- can be something like Dipper, i.e. information-state update

based..

- or FSA (specified in SCXML or similar?)

- but rules can be simple anyway, simple FSA-stuff:

- identification -> confirmation (repeat on negative) ->

orientation -> confirmation (repeat on negative) -> placement

(repeat on negative)

- S: "Welche Teil?" U: "Das zweite von links" S: "Das hier?"

U: "Nein, daneben". [ --> need to be able to deal with

context-dependent utterances ]

- do WOz pretty soon? Wizard hears user utterances, can trigger

simple prompts:

- "Welches Teil?" "Soll ich es drehen" "Wohin?";

"So?" "Hier?"

- to hide that Wizard is human, let GUI do mouse movements? I.e.,

wizard selects parameters of action (selecting piece, rotating

it, dragging it), then selects prompt ("So?"); this is then sent

to system which executes action (e.g., computes and executes

mouse path; plays synchronised utterance). This won't allow us

to test reaction to smooth turn-taking (since it is

non-incremental; the wizard will have to fully specify the

action), but it will allow us to test user reactions & learn

about the complexity of their speech. Especially the reactions to

CRs like "so?". E.g., "nein, eins weiter hoch".

050508cont

05/08/08 10:21

- semantics of utterances: for prototype, treat all referring

expressions as *names*, i.e. with their semantics specified in the

lexicon?

- for pieces: expressions like "the cross". A bit of a stretch to

treat things like "the second on the left side in the

second row" as names. This can be translated into some sort of

coordinate, but would still need to be resolved -- that this

expression refers to piece3 can't be a fact of the lexicon.

- for locations: expressions like "the back of the elephant" refer

to a location, which can initially be represented by a single

coordinate. Then zooming-in process sets in (see below).

- how much intelligence does the non-ling-action executor have? What

does the Dialogue Manager communicate to it?

Let's assume the question was "where shall I move it?" (i.e.,

piece has been identified), and the answer "to the back of the

elephant" has been correctly understood. This instruction is

underspecified. How do we get to the right location? What does the

DM tell the "arm"? And what does the arm answer back?

- DM: `move (10,20)'

- Arm: done. closest legal position is (11,20). [i.e., arm knows

about legal positions, identifies these independently]

- DM: to TTS `here?'

- User: `yes'

- DM to arm `drop'

- for prototype: movements can be interrupted by user. `action

barge-in'

- for prototype: nothing can be removed?

- TO DOs parser:

- brain-stormed a bit. (On Tuesday.)

- keep structure of referring expressions around, even if for

now we treat them as names.

- HPSG-inspired? With HEAD, DTRS, SEM, RESTR features etc.

- but basis (for semantics) is the frame:

ACTION: move

AGENS: system (implicit)

PATIENS: piece

START_POS:

TARGET_POS:

features can be left unspecified, even when utterance is

complete.

- identified issues:

1. complex descriptions (see above). Keep internal structure

of descriptions, for latter (concurrent) processing by

other modules

2. also need semantics for interaction management utterances

like "yes", "no", "one up", "daneben", etc.

3. need info about (potential) completeness. Parser features

as used in Nigh experiments.

4. fragments / context ellipsis.

"Welches Teil soll ich jetzt nehmen?" "Das Kreuz"

should produce same semantics as "Nimm das Kreuz"? Or is

this contextual merge done in later module?

- TO DOs for ASR part:

- get partial results w/ n-gram model?

- get confidence scores out

- learn more about configurations

- get n-best lists / lattices

---> Friday, 1st International Sphinx Day

minutes080508

05/08/08 10:21

- present: M, T, D

- priority: close processing chain. Finally get something from ASR

to parser to DM to TTS -- even if it is only a parrot system!

- Dialogue Manager:

- can be something like Dipper, i.e. information-state update

based..

- or FSA (specified in SCXML or similar?)

- but rules can be simple anyway, simple FSA-stuff:

- identification -> confirmation (repeat on negative) ->

orientation -> confirmation (repeat on negative) -> placement

(repeat on negative)

- S: "Welche Teil?" U: "Das zweite von links" S: "Das hier?"

U: "Nein, daneben". [ --> need to be able to deal with

context-dependent utterances ]

- do WOz pretty soon? Wizard hears user utterances, can trigger

simple prompts:

- "Welches Teil?" "Soll ich es drehen" "Wohin?";

"So?" "Hier?"

- to hide that Wizard is human, let GUI do mouse movements? I.e.,

wizard selects parameters of action (selecting piece, rotating

it, dragging it), then selects prompt ("So?"); this is then sent

to system which executes action (e.g., computes and executes

mouse path; plays synchronised utterance). This won't allow us

to test reaction to smooth turn-taking (since it is

non-incremental; the wizard will have to fully specify the

action), but it will allow us to test user reactions & learn

about the complexity of their speech. Especially the reactions to

CRs like "so?". E.g., "nein, eins weiter hoch".

050508cont

THE FRIGGING WIKI IS BROKEN. you can find the complete minutes on my weblog.

minutes080508

05/08/08 10:02

- present: M, T, D

- priority: close processing chain. Finally get something from ASR

to parser to DM to TTS -- even if it is only a parrot system!

- Dialogue Manager:

- can be something like Dipper, i.e. information-state update

based..

- or FSA (specified in SCXML or similar?)

- but rules can be simple anyway, simple FSA-stuff:

- identification -> confirmation (repeat on negative) ->

orientation -> confirmation (repeat on negative) -> placement

(repeat on negative)

- S: "Welche Teil?" U: "Das zweite von links" S: "Das hier?"

U: "Nein, daneben". [ --> need to be able to deal with

context-dependent utterances ]

- do WOz pretty soon? Wizard hears user utterances, can trigger

simple prompts:

- "Welches Teil?" "Soll ich es drehen" "Wohin?";

"So?" "Hier?"

- to hide that Wizard is human, let GUI do mouse movements? I.e.,

wizard selects parameters of action (selecting piece, rotating

it, dragging it), then selects prompt ("So?"); this is then sent

to system which executes action (e.g., computes and executes

mouse path; plays synchronised utterance). This won't allow us

to test reaction to smooth turn-taking (since it is

non-incremental; the wizard will have to fully specify the

action), but it will allow us to test user reactions & learn

about the complexity of their speech. Especially the reactions to

CRs like "so?". E.g., "nein, eins weiter hoch".

THE FRIGGING WIKI IS BROKEN. you can find the complete minutes on my weblog.