scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-09 22:31:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87

fix(e2e/ui): scope the drawer assertions to the visible, exact match

Browse source 
CI flagged the spec flaky twice more. Both were the same defect in different
places: the Logs drawer renders several nodes per string and the first in DOM
order is often hidden, so .first() waited 20s on an invisible element. The
entity assertions had a second problem on top, since getByText("EMAIL_ADDRESS")
substring-matched the masked prompt div, whose text contains <EMAIL_ADDRESS>,
rather than the entity chip.

Route every drawer assertion through onlyVisible, and match the entity type
and score exactly, which is what the panel renders them as: entity_type and
"Score: N.NN" each get their own span.

Verified on a live stack: 6 of 6 solo runs and the guardrails folder 5 of 5.
Mutating the analyzer to detect nothing turns the spec red on the raw address
reaching the spend log, so the assertions still carry their weight.
This commit is contained in:
Yuneng Jiang 2026-09-06 08:34:19 -07:00
parent 70f1894c28
commit fdd1c91096
No known key found for this signature in database
1 changed files with 10 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
tests/e2e/ui/tests/guardrails/presidioUserStory.spec.ts
View file

@ -137,17 +137,17 @@ test.describe("Presidio PII guardrail, end to end from the dashboard", () => {
await row.click();
const drawer = page.getByRole("dialog").first();
await expect(drawer.getByText("Guardrails & Policy Compliance")).toBeVisible({ timeout: 20_000 });
await expect(drawer.getByText(`Pre-call guardrail: ${guardrailName}`).first()).toBeVisible({ timeout: 20_000 });
const maskedPrompt = onlyVisible(drawer.getByText(`${marker}. Email me at <EMAIL_ADDRESS> or call <PHONE_NUMBER>.`));
await expect(maskedPrompt).toBeVisible({ timeout: 20_000 });
await expect(onlyVisible(drawer.getByText("Guardrails & Policy Compliance"))).toBeVisible({ timeout: 20_000 });
await expect(onlyVisible(drawer.getByText(`Pre-call guardrail: ${guardrailName}`))).toBeVisible({ timeout: 20_000 });
const maskedPrompt = drawer.getByText(`${marker}. Email me at <EMAIL_ADDRESS> or call <PHONE_NUMBER>.`);
await expect(onlyVisible(maskedPrompt)).toBeVisible({ timeout: 20_000 });
await drawer.getByText("2 matched").first().click();
await expect(drawer.getByText("Detected Entities (2)").first()).toBeVisible({ timeout: 10_000 });
await expect(drawer.getByText("EMAIL_ADDRESS").first()).toBeVisible({ timeout: 10_000 });
await expect(drawer.getByText("PHONE_NUMBER").first()).toBeVisible({ timeout: 10_000 });
await expect(drawer.getByText("Score: 1.00").first()).toBeVisible({ timeout: 10_000 });
await expect(drawer.getByText("Score: 0.75").first()).toBeVisible({ timeout: 10_000 });
await onlyVisible(drawer.getByText("2 matched")).click();
await expect(onlyVisible(drawer.getByText("Detected Entities (2)"))).toBeVisible({ timeout: 10_000 });
await expect(onlyVisible(drawer.getByText("EMAIL_ADDRESS", { exact: true }))).toBeVisible({ timeout: 10_000 });
await expect(onlyVisible(drawer.getByText("PHONE_NUMBER", { exact: true }))).toBeVisible({ timeout: 10_000 });
await expect(onlyVisible(drawer.getByText("Score: 1.00", { exact: true }))).toBeVisible({ timeout: 10_000 });
await expect(onlyVisible(drawer.getByText("Score: 0.75", { exact: true }))).toBeVisible({ timeout: 10_000 });
await expect(drawer.getByText(RAW_EMAIL)).toHaveCount(0);
await expect(drawer.getByText(RAW_PHONE)).toHaveCount(0);