Jerry Liu - Today we’re introducing ParseBench-2 - our most ambitious document pa

01 · Hook · 0:00–0:01

A man and a woman sit in white armchairs across a small plant table in a brick loft. The man holds a phone and gestures as he speaks, while the woman, a laptop on her lap, turns to answer. The camera holds a static medium two-shot.

Camera: Static

02 · Hook · 0:01–0:01

Two people sit in white armchairs in a brick-walled loft; a man in a black zip jacket faces a woman in a gray sweater and glasses who gestures while speaking, a laptop with stickers rests on her lap and a potted plant sits on a small table between them.

Camera: Static

03 · Hook · 0:01–0:02

A smiling woman with glasses sits in a cream armchair, glancing up and brushing her hair back, a laptop with two stickers on its lid open in front of her. A wooden bookshelf with books and trailing plants fills the background against a brick wall.

Camera: Static

04 · Product intro · 0:02–0:05

A light grid title card holds the LlamaIndex logo and a small gray tagline above a large black serif ParseBench-2, the 2 in a purple-blue gradient. The plain grid background then cross-dissolves into a soft live-action shot of two people seated in white chairs, while the title stays locked on top.

On-screen text: “LlamaIndex”, “THE OPEN BENCHMARK FOR DOCUMENT PARSING”, “ParseBench-2”

Camera: Static

05 · Product intro · 0:05–0:11

Two team members sit in white armchairs in a brick-walled loft, talking. White name cards with colored accent bars slide into the bottom corners, then a large white monospace title 'ParseBench-2' fades in over the center and out.

On-screen text: “Gauthami Polasani”, “FOUNDING PMM”, “Pierre-Loic Doulcet”, “MEMBER OF TECHNICAL STAFF”, “ParseBench-2”

Camera: Static

06 · Other · 0:11–0:14

A woman with glasses and a gray sweater sits in a white armchair holding a laptop, speaking in a room with a bookshelf, plants and a brick wall. She glances down at the screen as white captions appear on a dark bar along the bottom of the locked-off frame.

On-screen text: “We thought we'd do something different”, “so we are here with Pierre”

Camera: Static

07 · Other · 0:14–0:18

A man and a woman sit in white armchairs facing each other in a studio set with a brick wall, shelving and potted plants; the woman rests a laptop on her lap. The camera holds a locked-off medium shot while a subtitle line sits at the bottom.

On-screen text: “so we are here with Pierre”, “to understand the research and the thinking that went into ParseBench V2”

Camera: Static

08 · Problem · 0:18–0:22

A woman with glasses sits in a cream armchair, laptop on her lap, answering in a plant-filled home office. A white grid card on the left types out her question line by line, cornered by small pink, blue and orange squares. White captions track her speech along the bottom.

On-screen text: “What made you decide”, “What made you decide we needed an update for ParseBench?”, “we needed an update”, “for ParseBench?”, “Over the last year, AI is getting extremely good”

Techniques: Kinetic type

Camera: Static

09 · Problem · 0:22–0:27

A man in a black quilted jacket with a small pink llama emblem sits in a chair, hands clasped, speaking off-camera in a bright room with a palm plant and a shelved pot behind him. White subtitles run along the bottom as he talks.

On-screen text: “Over the last year, AI is getting extremely good”, “and progressing very fast on parsing documents”

Camera: Static

10 · Problem · 0:27–0:29

A man and a woman sit in white armchairs in a studio set with shelves and plants, facing each other in conversation. The woman holds a laptop with a llama sticker on her lap. A subtitle line stays at the bottom of the locked-off frame.

On-screen text: “We feel that most benchmarks start to be saturated”

Camera: Static

11 · Problem · 0:29–0:31

A cream paper-textured background holds a line chart. A fan of thin curves draws out from a vertical axis marked with small squares, while a large gradient numeral counts from 41.6 to 24.9 and a second value, 5.9, appears to the right. A caption sits at the bottom.

On-screen text: “We feel that most benchmarks start to be saturated”

Camera: Static

12 · Problem · 0:31–0:32

A paper-textured off-white frame holds a line chart labeled SCORE over TIME, with several thin curves rising and flattening toward a dashed 100 line. A gradient number counts down from 3.2 to 3.0 above a BEST - WORST label, and a caption line swaps at the bottom.

On-screen text: “We feel that most benchmarks start to be saturated”, “and don't give us useful signal anymore”

Camera: Static

13 · Problem · 0:32–0:34

A man in a black quilted zip jacket with a small pink llama emblem sits in a chair, hands clasped, talking to an off-camera interviewer. A tall palm plant and a shelf of potted plants sit behind him. The locked-off medium shot holds as white subtitles run along the bottom.

On-screen text: “and don't give us useful signal anymore”, “so it was time to build a new one”

Camera: Static

14 · Feature reveal · 0:34–0:37

A woman with glasses sits in a white chair with a laptop, answering an off-camera interviewer. A white grid card with small colored tabs slides in at the upper left, and the question fills in line by line. Subtitles run along the bottom.

On-screen text: “so it was time to build a new one”, “With V2, what were you trying to improve?”, “So with V2, we are trying to improve the benchmark accuracy”

Techniques: Kinetic type

Camera: Static

15 · Feature reveal · 0:37–0:40

A man in a black quilted jacket with a small pink llama emblem sits in a chair, speaking to an off-screen interviewer. The locked-off medium shot frames him against leafy green plants and a wooden shelf, with a white subtitle line at the bottom.

On-screen text: “So with V2, we are trying to improve the benchmark accuracy”

Camera: Static

16 · Feature reveal · 0:40–0:44

Two people sit in armchairs in a plant-filled room with a brick wall and shelves. The man on the left gestures while talking; the woman on the right holds a laptop on her lap. A white serif line builds word by word in the center as captions run along the bottom.

On-screen text: “So with V2, we are trying to improve the benchmark accuracy”, “in the sense that it should better reflect human judgment”, “Better reflect human judgment.”

Techniques: Kinetic type

Camera: Static

17 · Feature reveal · 0:44–0:46

A man in a black quilted jacket sits in a chair against a bright room with palm fronds and a shelved plant, talking and gesturing with both hands. The camera is locked off on a medium shot. A white caption bar reads his words along the bottom edge.

On-screen text: “To do that, we add a lot of documents to the benchmark”

Camera: Static

18 · Feature reveal · 0:46–0:50

Two people sit in white armchairs in a plant-filled room. The man on the left gestures while speaking, the woman on the right holds a laptop and listens. Subtitles appear at the bottom in a locked-off medium shot.

On-screen text: “To do that, we add a lot of documents to the benchmark”, “and we also spend a lot of effort on the metric”, “to make sure that whatever score comes out matches what a human will judge”

Camera: Static

19 · Feature reveal · 0:50–0:55

A man in a black quilted jacket sits in a bright, plant-filled room, gesturing as he speaks. A cream card on the left reveals the quote line by line, and a wavy line below it draws itself and fills with a rainbow gradient.

On-screen text: “to make sure that whatever score comes out matches what a human will judge”, “Whatever score”, “comes out matches”, “what a human”, “will judge.”

Techniques: Kinetic type

Camera: Static

20 · Feature reveal · 0:55–0:58

Two people sit in armchairs in a plant-filled studio, the woman holding a laptop. A white grid-paper card with pink, blue and orange tabs pops in at center; the question builds line by line in black serif, then the card exits. Subtitles run along the bottom.

On-screen text: “So when you say human judgment, what do you mean?”, “When you say”, “human judgment,”, “what do you mean?”, “For example, if you have a parser”

Techniques: Kinetic type

Camera: Static

21 · Feature reveal · 0:58–1:00

A man in a black zip jacket sits in a chair, gesturing with both hands as he speaks. He is held in a static medium shot against a leafy palm plant and a wooden shelf holding a potted plant.

On-screen text: “For example, if you have a parser”, “that would parse a page from a book”

Camera: Static

22 · Feature reveal · 1:00–1:06

A man in a black jacket talks on the left as a document page slides in from the right. Below it, the expected email appears in monospace, then the parser output is typed with two letters swapped and boxed in orange. A large serif line fades in at the end.

On-screen text: “that would parse a page from a book”, “EXPECTED”, “[email protected]”, “but decide to swap two letters in the email of the author”, “PARSER OUTPUT”, “[email protected]”, “Quite a critical mistake.”

Techniques: Split screen, Kinetic type

Camera: Pan

23 · Feature reveal · 1:06–1:11

Split screen: a man in a black jacket sits left, gesturing as he speaks, while a right panel shows a corporate document card, an expected versus parsed email with a highlighted typo, and a serif headline whose final word shifts from gray to black.

On-screen text: “it's quite a critical mistake”, “Quite a critical mistake.”, “because it's an important piece of information”

Techniques: Split screen

Camera: Static

24 · Feature reveal · 1:11–1:12

A man in a black jacket gestures as he speaks to a woman with glasses holding a sticker-covered laptop, both seated in white armchairs around a small plant table in a brick-walled loft. The shot is a static medium two-shot.

On-screen text: “And on most benchmarks or metrics”

Camera: Pan

25 · Feature reveal · 1:12–1:12

Two people sit in white armchairs facing each other in a plant-lined studio set. The man in a black jacket gestures as he speaks, the woman in gray holds a laptop on her lap. A white subtitle line is locked at the bottom of the frame.

On-screen text: “And on most benchmarks or metrics”

Camera: Static

26 · Feature reveal · 1:12–1:21

A white grid holds three labeled columns (TERMS, VENDOR, EMAIL); serif values appear, differing letters get boxed, then colored lines drop to reveal character counts that switch to Harmless, Minor, Critical. The scene cuts to a man in a black jacket speaking before plants.

On-screen text: “TERMS”, “Net 30”, “net 30”, “VENDOR”, “Acme Supply Co.”, “EMAIL”, “[email protected]”, “[email protected]”, “this will not get penalized a lot because it's just two letters getting swapped”, “1 CHARACTER / 1 CHARACTER / 2 CHARACTERS / HUMAN JUDGMENT / Harmless / Minor / Critical / whereas it's actually something that is very important for a human / ”

Techniques: Kinetic type

Camera: Static

27 · Feature reveal · 1:21–1:24

A man in a black jacket gestures while talking to a woman with a laptop on her lap, both seated in white armchairs before shelves and plants. The locked-off medium shot holds as captions roll along the bottom.

On-screen text: “So the way you automatically measure things”, “and the way you judge them as a human always have a gap”

Camera: Static

28 · Feature reveal · 1:24–1:28

A man in a black zip-up jacket with a small pink-and-blue llama emblem sits in a chair, speaking while gesturing with both hands. A locked-off medium shot frames him against a bright set with green plants and a wooden shelf.

On-screen text: “and the way you judge them as a human always have a gap”, “and we are trying to close this gap as much as possible”

Camera: Static

29 · Feature reveal · 1:28–1:30

A man in a black jacket speaks in a leafy office as a cream card on the left reads “Close this gap as much as possible.” with two blue bars whose orange gap shrinks shut. The shot cuts to a woman with glasses gesturing beside a sticker-covered laptop.

On-screen text: “Close this gap as much as possible.”, “and we are trying to close this gap as much as possible”, “How do you make sure an eval captures that human judgment?”

Camera: Static

30 · Feature reveal · 1:30–1:34

A woman in a grey sweater sits with a laptop on her lap, gesturing while speaking in a home office. On the left, a white grid card with colored squares scales up and fills in the question, then a thin white line draws across the shelf as the card exits.

On-screen text: “How do you make sure an eval captures that human judgment?”, “Yeah, so we spend quite a lot of effort on that”

Techniques: Kinetic type

Camera: Static

31 · Feature reveal · 1:34–1:35

A man in a black quilted zip-up with a small pink llama emblem sits in a bright, plant-filled office, hands clasped, answering a question. A question headline, a name tag, and a subtitle sit statically in the locked-off frame.

On-screen text: “How do you make sure an eval captures that human judgment?”, “MONCUR HOME”, “Llamalndex”, “Yeah, so we spend quite a lot of effort on that”

Camera: Static

32 · Feature reveal · 1:35–1:37

A woman with glasses and a gray sweater sits in a white armchair holding a silver laptop with a LlamaIndex sticker, gazing off camera. Behind her a brick wall, plants and a black-and-wood shelf. A white caption runs along the bottom edge.

On-screen text: “And for ParseBench V2, instead of directly designing the metric”

Camera: Static

33 · Feature reveal · 1:37–1:43

Locked-off medium shot of a man in a black zip jacket with a small pink llama patch, seated among plants, gesturing as he talks; bottom subtitles run, then a large white serif ~150 failure scenarios stat fades in at right.

On-screen text: “And for ParseBench V2, instead of directly designing the metric”, “we start by designing around 150 scenarios”, “failure scenarios”, “~150”

Camera: Static

34 · Feature reveal · 1:43–1:47

Two people sit in armchairs in a plant-filled room with brick walls and shelves. The man on the left gestures with both hands while speaking; the woman on the right listens, a laptop resting on her lap. A subtitle line sits at the bottom.

On-screen text: “that reflect how different things can go wrong or not in parsing”

Camera: Static

35 · Feature reveal · 1:47–1:49

A man in a black shirt with a small llama logo speaks in a bright room with plants and shelving, corner squares framing the shot. The frame then cuts to a white grid where a scatter of small black squares drifts into a loose horizontal cluster.

On-screen text: “that reflect how different things can go wrong or not in parsing”, “These scenarios were then rated for severity by humans”

Techniques: Particles

Camera: Static

36 · Feature reveal · 1:49–1:59

Hundreds of small colored squares drift as a loose scatter on a grid-paper background, then snap into four column blocks under the headers Critical, Major, Minor, Harmless. The dots re-sort into rows labeled Text, Formatting, Tables, Charts and Forms while the type counts fade out.

On-screen text: “These scenarios were then rated for severity by humans”, “and we use that to benchmark the metrics”, “and to see how the metric matches what a human would have judged”, “And I have to ask, how did LlamaParse perform?”

Techniques: Particles

Camera: Static

37 · Social proof · 1:59–2:02

A woman with glasses sits in a white armchair in a plant-filled loft, speaking in front of a laptop with a LlamaIndex sticker. A white grid card with small colored tabs slides in on the left, its question typed word by word. Subtitles at the bottom carry her answer.

On-screen text: “And I have to ask, how did LlamaParse perform?”, “How did LlamaParse perform?”, “We did quite well”, “Agentic Plus topped the benchmark”

Techniques: Kinetic type

Camera: Static

38 · Social proof · 2:02–2:09

A live-action strip of a man at a desk sits above a white grid panel that slides up to fill the frame. A serif headline and a llama line drawing hold on the left while a bar chart on the right cycles through five category tabs, bars extending and labels swapping. The panel then slides down to reveal the man again.

On-screen text: “LlamaParse tops every category.”, “Agentic Plus topped the benchmark”, “And this is on a benchmark that was not tailor-made to measure our accuracy”, “which is quite impressive”, “For someone evaluating parsers, what should they be looking at?”

Techniques: Masking, Split screen

Camera: Static

39 · Feature reveal · 2:09–2:13

A woman sits in a chair with a laptop, gesturing as she speaks. A white text box with a grid pattern appears on the left, and the question types out word by word.

On-screen text: “For someone evaluating parsers, what should they be looking at?”, “First of all, they should be looking at their own use case”

Techniques: Kinetic type

Camera: Static

40 · Feature reveal · 2:13–2:15

A man in a black jacket with a pink llama logo sits in a plant-filled office, gesturing with both hands as he talks. The shot cuts to a dim room where a large screen shows a grid of white squares, a person's head at the lower left.

On-screen text: “First of all, they should be looking at their own use case”, “and what the real need from parsing is”, “For someone evaluating”, “parsers, what should they”

Camera: Static

41 · Feature reveal · 2:15–2:17

On a pale grid backdrop, three empty outlined boxes sit side by side. A document page fills the left box, and the label “01 Table accuracy” appears beneath it while the bottom caption switches to a question about table accuracy, chart extraction, or text transcription.

On-screen text: “and what the real need from parsing is”, “01 Table accuracy”, “Is it table accuracy, chart extraction or precise text transcription?”

Camera: Static

42 · Feature reveal · 2:17–2:22

Three flat document pages, a table, a line chart and a legal text page, sit side by side on a light grid and fill in left to right. A numbered serif label sits under each, and a bottom caption bar swaps its sentence near the end.

On-screen text: “01 Table accuracy”, “Is it table accuracy, chart extraction or precise text transcription?”, “02 Chart extraction”, “03 Text transcription”, “And that's why in ParseBench we added different categories for all these things”

Camera: Static

43 · Feature reveal · 2:22–2:23

A man in a black jacket and a woman with a laptop sit in white armchairs across a small plant table, framed by shelves and greenery. He gestures with both hands while speaking; she listens. A white subtitle line sits locked at the bottom of the frame.

On-screen text: “And that's why in ParseBench we added different categories for all these things”

Camera: Static

44 · Feature reveal · 2:23–2:27

A white grid panel wipes up over a brief office shot, then a radial diagram draws from a center point: thin lines extend to small squares labeled TEXT, FORMATTING, TABLES, CHARTS and LAYOUT, and a faint polygon links the outer points.

On-screen text: “And that's why in ParseBench we added different categories for all these things”, “TEXT”, “FORMATTING”, “CHARTS”, “TABLES”, “LAYOUT”

Techniques: Masking

Camera: Static

45 · Feature reveal · 2:27–2:29

A radar chart on a light grid background settles its axis labels and extends a blue marker along the FORMS axis, then the shot cuts to a man in a black jacket speaking in front of a plant.

On-screen text: “and a satellite bench, for example, for financial tables”, “And of course, we are always continuing to improve our benchmarks”

Camera: Static

46 · Feature reveal · 2:29–2:34

A man in a black quilted zip jacket with a pink llama emblem sits in a bright office, palm plants blurred behind him. He gestures with both hands while speaking, framed in a locked-off medium close-up. White subtitles run along the bottom of the frame.

On-screen text: “And of course, we are always continuing to improve our benchmarks”, “so we will add more over time”

Camera: Static

47 · CTA · 2:34–2:35

A plain white frame holds a gray caption at the bottom. The caption scales away as a silver laptop with a Llamaindex sticker rises from the lower edge over a desk set with a plant and books, framed by small orange, blue and pink rectangles in the corners.

On-screen text: “so we will add more over time”

Techniques: Kinetic type

Camera: Static

48 · CTA · 2:35–2:35

A woman with glasses sits in a white armchair behind a laptop, speaking to camera in a brick-walled room with shelves and plants. A dark caption bar sits lower right. A white wipe clears the left edge at the start of the shot.

On-screen text: “ParseBench V2 is live”

Techniques: Masking

Camera: Static

49 · CTA · 2:35–2:42

A woman in glasses speaks behind a laptop in the right half, beside a white grid panel showing NOW LIVE and ParseBench-2 with colored corner squares. Subtitles track her speech below; the clip ends on a cut to a scan of an old handwritten recipe page.

On-screen text: “NOW LIVE”, “ParseBench-2”, “ParseBench V2 is live”, “and the research paper is also out for folks who are interested”, “We would love for you guys to try it out and let us know what you think!”

Techniques: Split screen

Camera: Static

50 · CTA · 2:42–2:43

A vintage railroad stock certificate with an engraved locomotive fills the frame on a black field, then cuts to a handwritten envelope with a red postmark and stamp on a gray field. Both documents sit flat and still.

On-screen text: “SHARES $100 EACH”

Camera: Static

51 · CTA · 2:43–2:43

A line-drawn typewriter keyboard with Japanese kana keys on cream paper wipes in a quick transition to a scanned open ledger with handwritten entries and round ink stamps on a blue background.

Techniques: Masking

Camera: Static

52 · CTA · 2:43–2:44

A vintage Chinese banknote, a pressed fern specimen, and a Russian sheet music page flash in turn on a textured cream paper background. A diagonal crosshatch pattern wipes over each frame as the images swap.

Camera: Static

53 · CTA · 2:44–2:44

A white grid background holds the LlamaIndex logo at top center, the label NOW LIVE, and the large serif word Parse. A thin box below types out the blog URL while small colored squares drift in the corners.

On-screen text: “NOW LIVE”, “Parse”, “llamaindex.ai/blog/introducing-parsebench-v2”

Camera: Static

54 · CTA · 2:44–2:47

On a light grid, the LlamaIndex logo and NOW LIVE caption sit above a large black serif headline ParseBench. A map image right of the word is swapped for a purple dash and gradient 2, forming ParseBench-2. Pixel clusters fade into the corners; a bordered monospace URL box sits below.

On-screen text: “NOW LIVE”, “ParseBench-2”, “llamaindex.ai/blog/introducing-parsebench-v2”

Camera: Static