01
Have a wonderful weekend and stay with the light!
Anonymous ACL submission
SeRV: Semantic-Aligned Residual Vector Quantization for American Sign Language Generation
Abstract
American Sign Language (ASL) generation remains challenging due to limited paired text-ASL motion data and the difficulty of learning motion representations both precise for reconstruction and predictable from linguistic input. Existing methods rely on motion tokenizers optimized for reconstruction, without explicit semantic supervision from paired text. As a result, the learned tokens remain limited in supporting semantically consistent and fine-grained ASL motion generation. To address this limitation, we propose SeRV (Semantic-Aligned Residual Vector Quantization), a semantic-aligned RVQ tokenizer for ASL generation. SeRV learns a semantically structured residual token space by combining sentence-level motion-text alignment with token-level text-conditioned supervision. Building on this tokenizer, a Hierarchical GPT predicts residual motion tokens in a coarse-to-fine manner, generating structurally coherent and semantically aligned 3D ASL motion. We further construct a large-scale reconstructed 3D ASL motion–text benchmark by recovering paired 3D motion from OpenASL videos. Experiments on both ASL datasets show that SeRV achieves state-of-the-art performance while producing semantically consistent 3D ASL motion directly from text.
Qualitative Results
Each clip shows one English sentence, written above it, signed in American Sign Language. On the left is the real signer (GT Video); the other panels are 3D avatars generated by SOKE, MaDiS, MotionGPT and SeRV (Ours) for the same sentence.
- GT Video
- SOKE
- MaDiS
- MotionGPT
- Ours
Clips play as you scroll to them. Use 0.5× to follow the hands more closely.
02
Hello everyone.
03
It was interesting because after we chatted, it started to feel strange because it became so quiet.
04
Chicago police do not have a suspect in custody and the investigation is ongoing.
05
See you tomorrow and stay with the light!
06
Sign1News.
07
Now we're ready to make the body of our spider.
08
Right now, we're trying to look at the Ultrabeat.
09
I'll start with my upper body even though my legs are going to be the ones that will really be the focus.
10
This cocktail is called Downshift.
Reconstructed 3D Motion
To build our reconstructed 3D ASL motion–text benchmark, we recover paired 3D motion from OpenASL videos. Each clip shows the real signer (GT Video) next to our reconstruction (GT Mesh).
01
And remember to like us on Facebook and follow us on Twitter.
02
Competitor.
03
Rest in peace Pete Frates.
04
Patterson then shook hands and said "My name is Michael Patterson, sir."
05
But now... they announced there will be no community captions.
06
We are not done yet.
07
The baby died.
08
I use these four to inspire them.
09
And it's still not over.
10
All indications point to a two-man race for the Democratic nomination between Biden and Bernie, and the winner will go up against President Trump.
11
There will be more information about the conference!
12
They are planning their next big march on March 8, 2018.
13
The sports are curling, alpine skiing, cross country skiing, snowboard, ice hockey, and chess.
14
Friends say Nicolai was in the midst of an ugly divorce and custody battle with her estranged husband.
15
Trump's political advisors hope he'll spotlight his accomplishments and 4 more years of promises kept.
16
Navigating through the topic of mental health comes with a lot of taboo and myths.
17
They eventually died in 1946.
18
In Florida police body cam footage shows the arrest of Tristan Scott Wix outside of a Daytona Beach grocery store.
19
Most importantly my mentor and friend.
20
The video of the trainer cutting Johnson's dreadlocks before victory went viral, and sparked outrage accusations of referee's racial bias.
21
They said the words, "without delay," was added to the legislation to make sure the work begins immediately.
22
I'm Melissa.
23
Gregory used to be a police officer and once worked for a local prosecutor.
24
See you tomorrow and stay with the light!
25
See you there!
26
The ADA requires that relay calls are treated the same as voice calls.
27
Hearing interpreters who have full access to English only use ASL up to a certain level.
28
As several states in the US have begun lifting stay-at-home restrictions, some countries are beginning to open up as well.
29
The Washington Post said the contract worth $1.3 billion means it would cost more than $30 million a mile.
30
If the committee approves the charges, ...