T0.2 - baseline captured; all six rendering defects confirmed present
Runs the app from a clean database, captures the before images, and records which of F1-F6 actually still reproduce. All six do. Rather than eyeball screenshots, each defect is measured in a browser by tests/f_items.py, which reports REPRODUCES / FIXED / INCONCLUSIVE and never a silent pass. That makes it both the wave 0 record and the wave 1-3 regression check: an item is done when its probe flips to FIXED. F1 hero says "Job A", app bar still says "Select a project", no reload F2 "Sign out" spans x382-432 in a 390px viewport - cut in half, 3 rows F3 chrome paints over the logo by 106x32px; .header-left collapses to 0 F4 comments drawer overlaps the header by 380x91px in the standalone creator F5 5 of 5 ENABLED wizard inputs compute #f4f4f4 on #e0e0e0 F6 11 cards in one 5,017px scroll, 0 tabs (review said ~4,700px; it grew) Three probes needed care to avoid reporting a false pass, and the traps are worth knowing before anyone verifies a fix: F1 disappears if localStorage is primed first, because then both sources of truth agree. The probe clears it and drives the real picker. F3 needs a long project name that is long IN THE DATABASE - any page reached with ?project= re-pulls it and overwrites a locally-faked one. It also cannot be measured by comparing .header-left to the chrome: under the long name .header-left (flex:1, min-width:0) collapses to clientWidth 0, so that comparison reports a tidy zero gap while the chrome paints across the logo. It measures against .logo, which is flex-shrink:0. My first two attempts at this probe both reported FIXED for those reasons; the screenshot did not. F5 must ignore genuinely disabled inputs or a fix looks done while real fields stay grey. 14 screenshots, not the 12 the plan asks for, because there are 7 pages (file-map D1). Capture also measures horizontal overflow, which is how BL-001 was found. Tooling: cdp.py gains viewport() and screenshot() - it could do neither, and T0.2 requires 390px and 1440px images. 390px sets the mobile flag rather than just narrowing the window, since every page declares width=device-width and Chrome otherwise lays out at 980px and no media query under test fires. Both new scripts reuse browser_check.py's seed() and start_server() instead of growing a second fixture. Existing browser_check still passes 71/71. No application code changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
86
docs/reference/baseline/README.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Baseline — August 14, 2026
|
||||
|
||||
**Task:** `T0.2` · **Branch:** `feat/wp-suite-r2-implementation` · commit before wave 1
|
||||
|
||||
The before images every later PR compares against, and the record of which rendering defects
|
||||
were confirmed present at the start.
|
||||
|
||||
## Screenshots
|
||||
|
||||
14 images, `<page>-<width>.png`, at 390px and 1440px. The plan says 12 (6 pages × 2); there are
|
||||
7 pages, so there are 14 — see `file-map.md` D1.
|
||||
|
||||
| Page | 390px | 1440px |
|
||||
|---|---|---|
|
||||
| login | `login-390.png` | `login-1440.png` |
|
||||
| launcher | `launcher-390.png` | `launcher-1440.png` |
|
||||
| SOP wizard | `sop-390.png` | `sop-1440.png` |
|
||||
| creator | `creator-390.png` | `creator-1440.png` |
|
||||
| admin | `admin-390.png` | `admin-1440.png` |
|
||||
| field view | `field-390.png` | `field-1440.png` |
|
||||
| directory/users | `users-390.png` | `users-1440.png` |
|
||||
|
||||
Regenerate, or capture the "after" half of a comparison:
|
||||
|
||||
```bash
|
||||
python tests/baseline_shots.py # -> here
|
||||
python tests/baseline_shots.py --out /tmp/after --label after
|
||||
```
|
||||
|
||||
390px is captured with Chrome's mobile flag set, not as a narrow desktop window. Every page
|
||||
declares `width=device-width`, so this is the layout a field tablet actually gets. The script
|
||||
asserts the width it asked for is the width the page saw.
|
||||
|
||||
## F1–F6: all six reproduce
|
||||
|
||||
Measured in a browser by `tests/f_items.py`, not read from source. Re-run any time:
|
||||
|
||||
```bash
|
||||
python tests/f_items.py # all six
|
||||
python tests/f_items.py F2 F5 # a subset, after one task
|
||||
```
|
||||
|
||||
Each probe reports `REPRODUCES`, `FIXED` or `INCONCLUSIVE` — never a silent pass. The same
|
||||
script is the regression check for waves 1–3: an item is done when its probe flips to `FIXED`.
|
||||
|
||||
| Item | Verdict | Measured | Evidence |
|
||||
|---|---|---|---|
|
||||
| **F1** | REPRODUCES | Picked "Job A" in the launcher picker: hero became `Job A`, app bar stayed `Select a project`, `wp_active_project=projA`, no reload. | `launcher-1440.png` |
|
||||
| **F2** | REPRODUCES | Field view at 390px: `Sign out` occupies x 382–432 against a 390px viewport — cut in half. Bar is 424px of content in 374px, ~3 rows tall. | `field-390.png`, `launcher-390.png` |
|
||||
| **F3** | REPRODUCES | SOP header with the real long name: injected chrome paints over the logo by 106×32px at 1024px and 41×32px at 1440px; `.header-left` collapses to `clientWidth 0`. | `f-evidence/F3-sop-header-*-longname.png` |
|
||||
| **F4** | REPRODUCES | Standalone creator: comments drawer overlaps the header by 380×91px once open. | `creator-1440.png` |
|
||||
| **F5** | REPRODUCES | 5 of 5 **enabled** wizard inputs compute to `rgb(244,244,244)` fill with `rgb(224,224,224)` border — the `#f4f4f4`/`#e0e0e0` the review named. | `sop-1440.png` |
|
||||
| **F6** | REPRODUCES | Creator is 11 cards in a single 5,017px scroll, 0 sectioning controls, 3 jump links. Review said ~4,700px; it has grown. | `creator-1440.png` |
|
||||
|
||||
### Notes that change how a fix gets verified
|
||||
|
||||
- **F1** is only visible if `localStorage` is *not* primed first. Setting both
|
||||
`wp_active_project` and `wp_active_project_obj` before load makes the two sources agree and
|
||||
hides the defect. The probe clears storage and drives the real picker.
|
||||
- **F3** is only visible with a genuinely long project name, and it has to be long **in the
|
||||
database** — any page reached with `?project=` re-pulls the project from the server and
|
||||
overwrites a name faked in `localStorage`. The probe seeds
|
||||
`Micron EUV Cleanroom Enable 2667008` as a real project.
|
||||
- **F3** cannot be measured by comparing `.header-left` to the chrome. Under the long name
|
||||
`.header-left` (`flex:1; min-width:0`) collapses to zero width, so that comparison reports a
|
||||
tidy zero gap while the chrome is painting across the logo. The probe measures against
|
||||
`.logo`, which is `flex-shrink:0` and therefore the one box in the bar whose position means
|
||||
something. A fix that leaves `.header-left` collapsed has not fixed F3.
|
||||
- **F5** must ignore genuinely disabled inputs, or the fix looks done while real fields stay
|
||||
grey. The probe counts only enabled, visible, non-hidden fields.
|
||||
|
||||
## Horizontal overflow, measured at capture time
|
||||
|
||||
`documentElement.scrollWidth` against `clientWidth`. Recorded because four pages overflow at
|
||||
390px and one also overflows at desk width, which no `F` item covers.
|
||||
|
||||
| Page | 390px viewport | 1440px viewport |
|
||||
|---|---|---|
|
||||
| launcher | 425px content | — |
|
||||
| SOP wizard | 429px content | — |
|
||||
| creator | 485px content | **1551px content** |
|
||||
| field view | 432px content | — |
|
||||
| login, admin, users | fits | fits |
|
||||
|
||||
The 1440px creator overflow is logged as `BL-001`. The 390px ones are `F2` and its
|
||||
neighbourhood, resolved properly by `B1` in wave 2.
|
||||
BIN
docs/reference/baseline/admin-1440.png
Normal file
|
After Width: | Height: | Size: 296 KiB |
BIN
docs/reference/baseline/admin-390.png
Normal file
|
After Width: | Height: | Size: 243 KiB |
BIN
docs/reference/baseline/creator-1440.png
Normal file
|
After Width: | Height: | Size: 340 KiB |
BIN
docs/reference/baseline/creator-390.png
Normal file
|
After Width: | Height: | Size: 299 KiB |
|
After Width: | Height: | Size: 35 KiB |
|
After Width: | Height: | Size: 39 KiB |
|
After Width: | Height: | Size: 25 KiB |
BIN
docs/reference/baseline/field-1440.png
Normal file
|
After Width: | Height: | Size: 38 KiB |
BIN
docs/reference/baseline/field-390.png
Normal file
|
After Width: | Height: | Size: 32 KiB |
BIN
docs/reference/baseline/launcher-1440.png
Normal file
|
After Width: | Height: | Size: 82 KiB |
BIN
docs/reference/baseline/launcher-390.png
Normal file
|
After Width: | Height: | Size: 81 KiB |
BIN
docs/reference/baseline/login-1440.png
Normal file
|
After Width: | Height: | Size: 23 KiB |
BIN
docs/reference/baseline/login-390.png
Normal file
|
After Width: | Height: | Size: 19 KiB |
BIN
docs/reference/baseline/sop-1440.png
Normal file
|
After Width: | Height: | Size: 46 KiB |
BIN
docs/reference/baseline/sop-390.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
BIN
docs/reference/baseline/users-1440.png
Normal file
|
After Width: | Height: | Size: 116 KiB |
BIN
docs/reference/baseline/users-390.png
Normal file
|
After Width: | Height: | Size: 80 KiB |
@@ -195,11 +195,16 @@ an addressable URL.
|
||||
|
||||
```bash
|
||||
uvicorn server.app:app # against a throwaway SQLite database
|
||||
python server/smoketest.py # API; signs in
|
||||
python tests/browser_check.py # pages boot, self-contained
|
||||
python server/smoketest.py # API; needs WP_SMOKE_USER / WP_SMOKE_PASSWORD
|
||||
python tests/browser_check.py # pages boot and render, self-contained
|
||||
python tests/baseline_shots.py # screenshots, self-contained
|
||||
python tests/f_items.py # does each of F1-F6 still reproduce?
|
||||
```
|
||||
|
||||
`server/smoketest.py` is the one that is **not** self-contained: it drives a server you point
|
||||
it at and aborts unless `WP_SMOKE_USER` and `WP_SMOKE_PASSWORD` are set, because every route
|
||||
but `/api/health` needs a session. Use an admin account — it creates and deletes a project.
|
||||
|
||||
`tests/browser_check.py` and `tests/baseline_shots.py` are self-contained: each creates a
|
||||
throwaway SQLite database, seeds a fixture, starts its own uvicorn on a free port, drives
|
||||
headless Edge or Chrome over CDP, and tears everything down. **Your real `wpsuite.db` is never
|
||||
@@ -217,6 +222,11 @@ python tests/baseline_shots.py --pages creator --widths 390,768,1024,1440
|
||||
regression test covers the clear-last-constraint path" without naming a home for it — both
|
||||
belong here.
|
||||
|
||||
`tests/f_items.py` is both halves of the same measurement: it recorded that all six defects
|
||||
reproduce before wave 1, and it is how waves 1–3 prove each one stopped. An item is done when
|
||||
its probe flips from `REPRODUCES` to `FIXED`. It never reports a silent pass — a probe that
|
||||
cannot decide says `INCONCLUSIVE`.
|
||||
|
||||
---
|
||||
|
||||
## Discrepancies
|
||||
|
||||
212
tests/baseline_shots.py
Normal file
@@ -0,0 +1,212 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Capture the suite's pages at both reference widths, for before/after comparison.
|
||||
|
||||
CLAUDE.md asks every frontend task to exercise the affected flow at 390px and at
|
||||
1440px and to put before and after screenshots in the PR. Doing that by hand 57
|
||||
times is how it stops getting done, so it is a script.
|
||||
|
||||
python tests/baseline_shots.py # -> docs/reference/baseline/
|
||||
python tests/baseline_shots.py --out /tmp/after # the "after" half of a diff
|
||||
python tests/baseline_shots.py --pages creator,field
|
||||
python tests/baseline_shots.py --widths 390,768,1024,1440
|
||||
|
||||
Self-contained, like tests/browser_check.py, whose seed() and start_server() it
|
||||
reuses rather than growing a second fixture: throwaway SQLite, its own uvicorn,
|
||||
headless Edge or Chrome over CDP, everything torn down afterwards. Your real
|
||||
database is never touched.
|
||||
|
||||
390px is emulated with the mobile flag set, not merely as a narrow desktop window.
|
||||
Every page in html/ declares width=device-width, so this is the layout a field
|
||||
tablet actually gets; without the flag Chrome lays out at 980px and the media
|
||||
queries under test never fire. The script asserts the width it asked for is the
|
||||
width the page saw, because that failure is otherwise invisible in a PNG.
|
||||
|
||||
Exit codes: 0 all captured · 1 one or more pages failed · 2 could not run.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import cdp # noqa: E402
|
||||
from browser_check import seed, start_server # noqa: E402
|
||||
|
||||
# Page copy contains em dashes and other non-cp1252 characters, and the default
|
||||
# Windows console encoding raises UnicodeEncodeError on them mid-run.
|
||||
try:
|
||||
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||
except (AttributeError, ValueError): # pragma: no cover
|
||||
pass
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
DEFAULT_OUT = os.path.join(REPO, "docs", "reference", "baseline")
|
||||
|
||||
# The active project is seeded as Job A. project-data.js keeps the id and a
|
||||
# denormalised copy under these two keys; both are set, because a page that reads
|
||||
# only the object would otherwise render its empty state.
|
||||
ACTIVE_ID = "projA"
|
||||
ACTIVE_OBJ = {"id": "projA", "name": "Job A", "number": "A-1", "client": "Internal QA"}
|
||||
|
||||
# name, file, user whose session to use, JS that means "this page has its data".
|
||||
# login is visited signed OUT — it is the one page whose real state is no session.
|
||||
PAGES = [
|
||||
("login", "login.html", None, None),
|
||||
("launcher", "index.html", "root", "!!document.querySelector('body')"),
|
||||
("sop", "work-package-suite.html", "root", "!!document.querySelector('.header-left, header')"),
|
||||
("creator", "wp-creation-index.html", "root", "!!document.querySelector('#wp_number, .field')"),
|
||||
("admin", "admin.html", "root", "!!document.querySelector('main, .card')"),
|
||||
("field", "field.html", "root", "!!document.querySelector('body')"),
|
||||
("users", "users.html", "root",
|
||||
"!!document.querySelector('#users-table table, #users-table .note:not(:empty)')"),
|
||||
]
|
||||
|
||||
_OK, _BAD, _OVERFLOW = [], [], []
|
||||
|
||||
|
||||
def _c(s, code):
|
||||
return f"\033[{code}m{s}\033[0m" if sys.stdout.isatty() else s
|
||||
|
||||
|
||||
def capture(page, base, tok, name, filename, user, wait_for, widths, out, label):
|
||||
"""Shoot one page at every width. Returns True if all of them landed."""
|
||||
ok = True
|
||||
page.clear_cookies()
|
||||
if user:
|
||||
page.set_cookie("wp_session", tok[user])
|
||||
|
||||
# localStorage is per-origin, so prime it once on this origin before the real
|
||||
# navigation. Signed-out login.html is left alone: giving it an active project
|
||||
# would be staging a state that page never has.
|
||||
if user:
|
||||
page.goto(base + "/index.html")
|
||||
page.eval(
|
||||
f"localStorage.setItem('wp_active_project', {ACTIVE_ID!r});"
|
||||
f"localStorage.setItem('wp_active_project_obj', {json.dumps(ACTIVE_OBJ)!r});"
|
||||
"true")
|
||||
|
||||
for w in widths:
|
||||
page.viewport(w, 900, mobile=(w <= 500))
|
||||
page.goto(base + "/" + filename, wait_for=wait_for)
|
||||
time.sleep(0.5) # webfonts and late-injected chrome
|
||||
|
||||
# Ask the page how wide it actually ended up. A page whose content will not
|
||||
# fit forces the initial containing block wider than the device, so innerWidth
|
||||
# comes back above what was requested and everything in the shot is at the
|
||||
# wrong scale. That is a finding about the page, not a failure of the capture,
|
||||
# so it is measured and reported and the screenshot is still taken.
|
||||
m = page.eval(
|
||||
"JSON.stringify({inner: window.innerWidth,"
|
||||
" scroll: document.documentElement.scrollWidth,"
|
||||
" client: document.documentElement.clientWidth})")
|
||||
m = json.loads(m)
|
||||
overflow = m["inner"] != w or m["scroll"] > m["client"]
|
||||
if overflow:
|
||||
_OVERFLOW.append(
|
||||
f"{name}@{w}: laid out {m['inner']}px, content {m['scroll']}px "
|
||||
f"in a {m['client']}px viewport")
|
||||
|
||||
suffix = f"-{label}" if label else ""
|
||||
path = os.path.join(out, f"{name}-{w}{suffix}.png")
|
||||
try:
|
||||
page.screenshot(path)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
_BAD.append(f"{name}@{w}")
|
||||
print(" " + _c("FAIL", "31") + f" {name} {w}px — {exc}")
|
||||
ok = False
|
||||
continue
|
||||
|
||||
errs = page.js_errors()
|
||||
size = os.path.getsize(path)
|
||||
_OK.append(f"{name}@{w}")
|
||||
flags = []
|
||||
if overflow:
|
||||
flags.append(f"overflows: {m['scroll']}px of content")
|
||||
if errs:
|
||||
flags.append(f"{len(errs)} JS error(s)")
|
||||
note = " " + " · ".join(flags) if flags else ""
|
||||
print(" " + _c("OK", "32") + f" {name:9s} {w:>4}px {size:>7,}b "
|
||||
f"{os.path.basename(path)}{_c(note, '33')}")
|
||||
for e in errs[:3]:
|
||||
print(f" {_c('js:', '33')} {e[:110]}")
|
||||
return ok
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--out", default=DEFAULT_OUT, help="directory for the PNGs")
|
||||
ap.add_argument("--widths", default="390,1440", help="comma-separated CSS widths")
|
||||
ap.add_argument("--pages", default="", help="comma-separated subset of page names")
|
||||
ap.add_argument("--label", default="", help="suffix, e.g. --label after")
|
||||
ap.add_argument("--base-url", default="", help="use a server that is already up")
|
||||
args = ap.parse_args()
|
||||
|
||||
widths = [int(w) for w in args.widths.split(",") if w.strip()]
|
||||
wanted = {p.strip() for p in args.pages.split(",") if p.strip()}
|
||||
pages = [p for p in PAGES if not wanted or p[0] in wanted]
|
||||
if wanted - {p[0] for p in PAGES}:
|
||||
print(f"unknown page(s): {', '.join(sorted(wanted - {p[0] for p in PAGES}))}")
|
||||
print(f"known: {', '.join(p[0] for p in PAGES)}")
|
||||
return 2
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
|
||||
if not cdp.find_browser():
|
||||
print("no headless-capable browser found (set WP_BROWSER)")
|
||||
return 2
|
||||
|
||||
tmpdir = tempfile.mkdtemp(prefix="wpsuite-baseline-")
|
||||
db_path = os.path.join(tmpdir, "baseline.db")
|
||||
proc = browser = None
|
||||
try:
|
||||
tok = seed(db_path)
|
||||
if args.base_url:
|
||||
base = args.base_url.rstrip("/")
|
||||
else:
|
||||
port = cdp.free_port()
|
||||
proc = start_server(port, db_path)
|
||||
base = f"http://127.0.0.1:{port}"
|
||||
if not proc:
|
||||
print("the server would not start")
|
||||
return 2
|
||||
|
||||
print(f"\n {len(pages)} page(s) x {len(widths)} width(s) -> {args.out}\n")
|
||||
browser = cdp.Browser()
|
||||
page = browser.page()
|
||||
for name, filename, user, wait_for in pages:
|
||||
capture(page, base, tok, name, filename, user, wait_for,
|
||||
widths, args.out, args.label)
|
||||
|
||||
print(f"\n {_c(str(len(_OK)) + ' captured', '32')}"
|
||||
+ (f", {_c(str(len(_BAD)) + ' failed', '31')}" if _BAD else ""))
|
||||
if _OVERFLOW:
|
||||
print(f"\n {_c('horizontal overflow', '33')} "
|
||||
f"({len(_OVERFLOW)} of {len(_OK)} shots) — content wider than the "
|
||||
f"viewport it was asked for:")
|
||||
for line in _OVERFLOW:
|
||||
print(f" {line}")
|
||||
print()
|
||||
return 1 if _BAD else 0
|
||||
finally:
|
||||
if browser:
|
||||
browser.close()
|
||||
if proc:
|
||||
proc.kill()
|
||||
proc.wait(timeout=10)
|
||||
# seed() built an engine in this process too; drop it before deleting the
|
||||
# file or Windows keeps the handle open. Same reason as browser_check.
|
||||
try:
|
||||
from server.db import engine
|
||||
engine.dispose()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
42
tests/cdp.py
@@ -296,6 +296,48 @@ class Page:
|
||||
self.eval(f"document.dispatchEvent(new KeyboardEvent('keydown',{{key:{name!r}}}))")
|
||||
time.sleep(settle)
|
||||
|
||||
def viewport(self, width, height=900, mobile=False, scale=1):
|
||||
"""Emulate a viewport width. The two that matter are 390 (the gloved-hands
|
||||
field tablet, where the worst rendering was found) and 1440 (the desk).
|
||||
|
||||
`mobile` also sets the mobile flag and a meta-viewport-aware layout, which
|
||||
is what a tablet actually reports; without it a 390px-wide desktop window
|
||||
is not the same test."""
|
||||
self.ws.call("Emulation.setDeviceMetricsOverride", {
|
||||
"width": int(width), "height": int(height),
|
||||
"deviceScaleFactor": scale, "mobile": bool(mobile),
|
||||
})
|
||||
time.sleep(0.35) # let media queries and reflow settle
|
||||
self.ws.drain(0.2)
|
||||
return self
|
||||
|
||||
def screenshot(self, path, full_page=True):
|
||||
"""Write a PNG to `path`, creating parent directories. Returns the path.
|
||||
|
||||
full_page captures the whole document rather than the visible box: the
|
||||
creator form is roughly 4,700px tall and a viewport-sized shot of it would
|
||||
hide the thing being compared. Chrome refuses beyond 16,384px, so an
|
||||
over-tall page is clamped rather than failing the capture."""
|
||||
params = {"format": "png"}
|
||||
if full_page:
|
||||
try:
|
||||
m = self.ws.call("Page.getLayoutMetrics")
|
||||
size = m.get("cssContentSize") or m.get("contentSize") or {}
|
||||
w, h = size.get("width"), size.get("height")
|
||||
if w and h:
|
||||
params["clip"] = {"x": 0, "y": 0, "width": w,
|
||||
"height": min(h, 16384), "scale": 1}
|
||||
params["captureBeyondViewport"] = True
|
||||
except (RuntimeError, TimeoutError):
|
||||
pass # fall back to a viewport-sized shot
|
||||
shot = self.ws.call("Page.captureScreenshot", params, timeout=45)
|
||||
parent = os.path.dirname(os.path.abspath(path))
|
||||
if parent:
|
||||
os.makedirs(parent, exist_ok=True)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(base64.b64decode(shot["data"]))
|
||||
return path
|
||||
|
||||
def js_errors(self):
|
||||
"""Everything that means 'this page did not boot cleanly': uncaught
|
||||
exceptions, console.error calls, and browser-logged errors.
|
||||
|
||||
422
tests/f_items.py
Normal file
@@ -0,0 +1,422 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does each of F1-F6 still reproduce? Measured in a browser, not read from source.
|
||||
|
||||
T0.2 has to record reproduces / does not reproduce for every rendering defect before
|
||||
wave 1 starts fixing them, and waves 1-3 have to prove each one stopped. Both halves
|
||||
are the same measurement, so this is one script that reports either way:
|
||||
|
||||
REPRODUCES the defect is present -- correct before its fix lands
|
||||
FIXED the defect is gone -- correct after
|
||||
INCONCLUSIVE the probe could not decide; never silently a pass
|
||||
|
||||
python tests/f_items.py # all six
|
||||
python tests/f_items.py F2 F5 # a subset, e.g. after one task
|
||||
|
||||
Self-contained on the same terms as tests/browser_check.py, whose seed() and
|
||||
start_server() it reuses: throwaway SQLite, its own uvicorn, headless Edge or
|
||||
Chrome, all torn down afterwards. Your real database is never touched.
|
||||
|
||||
Each probe states the threshold it applies, because "the drawer is off-screen" and
|
||||
"the fields look disabled" are judgements a screenshot cannot settle in CI. Where the
|
||||
review gave a number, that number is the threshold.
|
||||
|
||||
Exit codes: 0 ran to completion (read the report) - 2 could not run.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import cdp # noqa: E402
|
||||
from browser_check import seed, start_server # noqa: E402
|
||||
|
||||
# Project names and page copy contain em dashes and other non-cp1252 characters, and
|
||||
# the default Windows console encoding raises UnicodeEncodeError on them mid-run.
|
||||
try:
|
||||
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||
except (AttributeError, ValueError): # pragma: no cover
|
||||
pass
|
||||
|
||||
REPRO, FIXED, UNKNOWN = "REPRODUCES", "FIXED", "INCONCLUSIVE"
|
||||
_RESULTS = []
|
||||
|
||||
# F3 only breaks with a real project name. "Micron EUV Cleanroom Enable 2667008" is
|
||||
# the one the review used and the one that overflows the header.
|
||||
LONG_ID = "projLong"
|
||||
LONG_NAME = "Micron EUV Cleanroom Enable 2667008"
|
||||
|
||||
|
||||
def add_long_project(db_path):
|
||||
"""Add the long-named project to the seeded database, before the server starts.
|
||||
|
||||
It has to exist in the database rather than be faked in localStorage: every page
|
||||
reached with ?project= re-pulls the project from the server, which would overwrite
|
||||
a local fake and quietly measure the short name instead -- a false FIXED."""
|
||||
os.environ["DATABASE_URL"] = "sqlite:///" + db_path.replace("\\", "/")
|
||||
from server.db import SessionLocal
|
||||
from server import models
|
||||
with SessionLocal() as db:
|
||||
db.add(models.Project(id=LONG_ID, name=LONG_NAME, number="2667008",
|
||||
client="Micron"))
|
||||
db.flush()
|
||||
db.add(models.ProjectMember(id="pmLong", user_id="user_root",
|
||||
project_id=LONG_ID, role=""))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _c(s, code):
|
||||
return f"\033[{code}m{s}\033[0m" if sys.stdout.isatty() else s
|
||||
|
||||
|
||||
def report(item, verdict, detail):
|
||||
colour = {REPRO: "31", FIXED: "32", UNKNOWN: "33"}[verdict]
|
||||
_RESULTS.append((item, verdict, detail))
|
||||
print(f" {item:3s} {_c(verdict, colour):22s} {detail}")
|
||||
|
||||
|
||||
def rect(page, sel):
|
||||
"""Bounding box of the first match, or None. Zero-size counts as absent."""
|
||||
got = page.eval(
|
||||
f"(function(){{var e=document.querySelector({sel!r});if(!e)return 'null';"
|
||||
"var r=e.getBoundingClientRect();"
|
||||
"return JSON.stringify({x:r.x,y:r.y,w:r.width,h:r.height,"
|
||||
"right:r.right,bottom:r.bottom,text:(e.textContent||'').trim().slice(0,60)});})()")
|
||||
if not got or got == "null":
|
||||
return None
|
||||
r = json.loads(got)
|
||||
return None if r["w"] == 0 and r["h"] == 0 else r
|
||||
|
||||
|
||||
# ── F1 ────────────────────────────────────────────────────────────────────────
|
||||
def f1(page, base, tok):
|
||||
"""App bar stale while the page body shows the active project.
|
||||
|
||||
Driven through the real control: land on the launcher with storage cleared, pick a
|
||||
project from the picker, and read both labels without reloading. That is the
|
||||
interaction the review describes -- the hero and the picker update, the app bar
|
||||
does not, because wp-chrome.js renders projectLabel() once and only refreshes it
|
||||
on the /api/projects callback, never on the selection itself.
|
||||
|
||||
Priming localStorage before load would hide this, so it is cleared first."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440)
|
||||
page.goto(base + "/index.html")
|
||||
page.eval("localStorage.clear(); true")
|
||||
page.goto(base + "/index.html", wait_for="!!document.querySelector('.wpc-proj-name')")
|
||||
time.sleep(1.3) # let /api/projects land and wpcRefresh run
|
||||
|
||||
picked = page.eval("""(function(){
|
||||
var s=document.querySelector('select'); if(!s) return 'no-select';
|
||||
var o=[].slice.call(s.options).filter(function(x){return x.value==='projA';})[0];
|
||||
if(!o) return 'no-projA';
|
||||
s.value='projA'; s.dispatchEvent(new Event('change',{bubbles:true}));
|
||||
return 'ok';})()""")
|
||||
if picked != "ok":
|
||||
return report("F1", UNKNOWN, f"could not drive the project picker ({picked})")
|
||||
time.sleep(1.4) # generous: a correct fix may re-render async
|
||||
|
||||
bar = rect(page, ".wpc-proj-name")
|
||||
hero = rect(page, "#hero-title")
|
||||
if not bar or not hero:
|
||||
return report("F1", UNKNOWN, "app bar or hero not found after selection")
|
||||
stored = page.eval("localStorage.getItem('wp_active_project')")
|
||||
reloaded = page.eval("location.search")
|
||||
|
||||
bar_txt, hero_txt = bar["text"], hero["text"]
|
||||
body_knows = "Job A" in hero_txt
|
||||
bar_knows = "Job A" in bar_txt
|
||||
if body_knows and not bar_knows:
|
||||
return report("F1", REPRO,
|
||||
f"picked Job A: hero {hero_txt!r}, app bar still {bar_txt!r} "
|
||||
f"(wp_active_project={stored!r}, no reload{', url ' + reloaded if reloaded else ''})")
|
||||
if not body_knows:
|
||||
return report("F1", UNKNOWN,
|
||||
f"selection did not reach the hero either (hero {hero_txt!r}) - probe drove the wrong control")
|
||||
return report("F1", FIXED,
|
||||
f"app bar {bar_txt!r} tracks hero {hero_txt!r} in the same interaction")
|
||||
|
||||
|
||||
# ── F2 ────────────────────────────────────────────────────────────────────────
|
||||
def f2(page, base, tok):
|
||||
"""App bar clips at 390px. The review names Sign out cut in half and search
|
||||
truncated, on the field view. Threshold: any bar control whose box crosses the
|
||||
viewport edge, or a bar that scrolls wider than it is."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(390, 900, mobile=True)
|
||||
page.goto(base + "/field.html")
|
||||
page.eval("localStorage.setItem('wp_active_project','projA');"
|
||||
"localStorage.setItem('wp_active_project_obj',"
|
||||
"JSON.stringify({id:'projA',name:'Job A',number:'A-1'})); true")
|
||||
page.goto(base + "/field.html", wait_for="!!document.querySelector('.wp-appbar')")
|
||||
time.sleep(0.8)
|
||||
|
||||
over = page.eval("""(function(){
|
||||
var bar=document.querySelector('.wp-appbar'); if(!bar) return 'null';
|
||||
var vw=document.documentElement.clientWidth, out=[];
|
||||
bar.querySelectorAll('a,button,input').forEach(function(e){
|
||||
var r=e.getBoundingClientRect();
|
||||
if(r.width===0&&r.height===0) return;
|
||||
if(r.right>vw+0.5||r.left<-0.5)
|
||||
out.push(((e.textContent||e.placeholder||e.tagName).trim().slice(0,26))
|
||||
+' @'+Math.round(r.left)+'-'+Math.round(r.right));
|
||||
});
|
||||
var br=bar.getBoundingClientRect();
|
||||
return JSON.stringify({vw:vw, rows:Math.round(br.height/48), clipped:out,
|
||||
scrollW:bar.scrollWidth, clientW:bar.clientWidth});})()""")
|
||||
if over == "null":
|
||||
return report("F2", UNKNOWN, ".wp-appbar not found on field.html")
|
||||
m = json.loads(over)
|
||||
if m["clipped"] or m["scrollW"] > m["clientW"]:
|
||||
why = f"{len(m['clipped'])} control(s) past the {m['vw']}px edge"
|
||||
if m["clipped"]:
|
||||
why += ": " + "; ".join(m["clipped"][:3])
|
||||
if m["scrollW"] > m["clientW"]:
|
||||
why += f" | bar {m['scrollW']}px in {m['clientW']}px"
|
||||
return report("F2", REPRO, why + f" | ~{m['rows']} row(s) tall")
|
||||
return report("F2", FIXED, f"no control past {m['vw']}px, bar {m['rows']} row(s)")
|
||||
|
||||
|
||||
# ── F3 ────────────────────────────────────────────────────────────────────────
|
||||
def f3(page, base, tok):
|
||||
""".header-left and the injected chrome overlap in the SOP header. Verified with
|
||||
the real long project name, which is what the review says breaks it. Threshold:
|
||||
the two boxes intersect by more than 2px on both axes."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440)
|
||||
# Measure against the LOGO, not against .header-left. With the long name
|
||||
# .header-left (flex:1, min-width:0) collapses to clientWidth 0, so comparing its
|
||||
# box to the chrome's reports a tidy zero gap while the chrome is in fact painting
|
||||
# straight across the logo. The logo is flex-shrink:0, so it is the one box in the
|
||||
# bar whose position means something.
|
||||
hits, measured, clear = [], 0, []
|
||||
for w in (390, 768, 1024, 1440):
|
||||
page.viewport(w, 900, mobile=(w <= 500))
|
||||
page.goto(base + f"/work-package-suite.html?project={LONG_ID}",
|
||||
wait_for="!!document.querySelector('.header-left')")
|
||||
time.sleep(0.9)
|
||||
got = page.eval("""(function(){
|
||||
var hl=document.querySelector('.header-left'),
|
||||
logo=document.querySelector('.header-left .logo'),
|
||||
ch=document.querySelector('.wpc-proj')||document.querySelector('.wp-chrome');
|
||||
if(!hl||!logo||!ch) return 'null';
|
||||
var L=logo.getBoundingClientRect(), C=ch.getBoundingClientRect();
|
||||
var ox=Math.min(C.right,L.right)-Math.max(C.left,L.left);
|
||||
var oy=Math.min(C.bottom,L.bottom)-Math.max(C.top,L.top);
|
||||
return JSON.stringify({
|
||||
ox:Math.round(ox), oy:Math.round(oy),
|
||||
collapsed: hl.clientWidth===0 && hl.scrollWidth>0,
|
||||
hlsw:hl.scrollWidth, hlcw:hl.clientWidth,
|
||||
name:((document.querySelector('.wpc-proj-name')||{}).textContent||'').trim()});})()""")
|
||||
if got == "null":
|
||||
continue
|
||||
measured += 1
|
||||
o = json.loads(got)
|
||||
why = []
|
||||
if o["ox"] > 1 and o["oy"] > 1:
|
||||
why.append(f"chrome over the logo by {o['ox']}x{o['oy']}px")
|
||||
if o["collapsed"]:
|
||||
why.append(f".header-left collapsed to 0 (content {o['hlsw']}px)")
|
||||
if why:
|
||||
hits.append(f"{w}px: " + ", ".join(why))
|
||||
else:
|
||||
clear.append(f"{w}px")
|
||||
if not measured:
|
||||
return report("F3", UNKNOWN,
|
||||
".header-left, its logo and the injected chrome were never all present")
|
||||
if hits:
|
||||
return report("F3", REPRO, "; ".join(hits))
|
||||
return report("F3", FIXED,
|
||||
f"logo clear of the chrome and .header-left intact at {', '.join(clear)} "
|
||||
f"with the long project name")
|
||||
|
||||
|
||||
# ── F4 ────────────────────────────────────────────────────────────────────────
|
||||
def f4(page, base, tok):
|
||||
"""Comments drawer renders off-screen and over the header in the STANDALONE
|
||||
creator (no ?embedded=1). Threshold: once open, the drawer's box sits wholly or
|
||||
partly outside the viewport, or intersects the header."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440)
|
||||
page.goto(base + "/wp-creation-index.html?project=projA",
|
||||
wait_for="!!document.querySelector('#cmt-drawer')")
|
||||
time.sleep(0.8)
|
||||
before = rect(page, "#cmt-drawer")
|
||||
if not before:
|
||||
return report("F4", UNKNOWN, "#cmt-drawer not found")
|
||||
try:
|
||||
page.click("#comments-btn", settle=0.9)
|
||||
except Exception:
|
||||
page.eval("typeof toggleComments==='function'?(toggleComments(),true):false")
|
||||
time.sleep(0.9)
|
||||
|
||||
got = page.eval("""(function(){
|
||||
var d=document.querySelector('#cmt-drawer'); if(!d) return 'null';
|
||||
var r=d.getBoundingClientRect();
|
||||
var vw=document.documentElement.clientWidth, vh=document.documentElement.clientHeight;
|
||||
var open=d.classList.contains('open')||d.getAttribute('aria-hidden')==='false';
|
||||
var hdr=document.querySelector('.wp-appbar,.header,header');
|
||||
var ov=null;
|
||||
if(hdr){var h=hdr.getBoundingClientRect();
|
||||
var ox=Math.min(r.right,h.right)-Math.max(r.left,h.left);
|
||||
var oy=Math.min(r.bottom,h.bottom)-Math.max(r.top,h.top);
|
||||
if(ox>2&&oy>2) ov=Math.round(ox)+'x'+Math.round(oy);}
|
||||
return JSON.stringify({open:open,x:Math.round(r.x),right:Math.round(r.right),
|
||||
w:Math.round(r.width),vw:vw,vh:vh,offscreen:(r.right>vw+2||r.x<-2),overHeader:ov});})()""")
|
||||
if got == "null":
|
||||
return report("F4", UNKNOWN, "#cmt-drawer vanished after toggle")
|
||||
d = json.loads(got)
|
||||
if not d["open"]:
|
||||
return report("F4", UNKNOWN, "drawer did not open; cannot judge placement")
|
||||
bad = []
|
||||
if d["offscreen"]:
|
||||
bad.append(f"box {d['x']}-{d['right']} outside a {d['vw']}px viewport")
|
||||
if d["overHeader"]:
|
||||
bad.append(f"overlaps the header by {d['overHeader']}px")
|
||||
if bad:
|
||||
return report("F4", REPRO, "standalone creator: " + "; ".join(bad))
|
||||
return report("F4", FIXED, f"drawer fully on screen ({d['x']}-{d['right']} of {d['vw']}px), clear of the header")
|
||||
|
||||
|
||||
# ── F5 ────────────────────────────────────────────────────────────────────────
|
||||
def f5(page, base, tok):
|
||||
"""SOP wizard inputs read as read-only. The review names the exact colours:
|
||||
#f4f4f4 fill, #e0e0e0 border, because the wizard never sees --cds-field #ffffff.
|
||||
Threshold: an ENABLED input computing to that fill."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440)
|
||||
page.goto(base + "/work-package-suite.html?project=projA",
|
||||
wait_for="!!document.querySelector('.field input, .field select')")
|
||||
time.sleep(0.9)
|
||||
got = page.eval("""(function(){
|
||||
var out=[], n=0;
|
||||
document.querySelectorAll('.field input,.field select,.field textarea').forEach(function(e){
|
||||
if(e.disabled||e.readOnly||e.type==='hidden'||e.type==='checkbox'||e.type==='radio') return;
|
||||
var r=e.getBoundingClientRect(); if(r.width===0&&r.height===0) return;
|
||||
n++;
|
||||
var s=getComputedStyle(e);
|
||||
out.push(s.backgroundColor+' | '+s.borderTopColor);
|
||||
});
|
||||
var tally={}; out.forEach(function(v){tally[v]=(tally[v]||0)+1;});
|
||||
return JSON.stringify({n:n,tally:tally});})()""")
|
||||
m = json.loads(got)
|
||||
if not m["n"]:
|
||||
return report("F5", UNKNOWN, "no enabled wizard inputs found")
|
||||
GREY = ("rgb(244, 244, 244)", "rgb(243, 243, 243)")
|
||||
greyed = sum(c for k, c in m["tally"].items() if any(g in k for g in GREY))
|
||||
top = sorted(m["tally"].items(), key=lambda kv: -kv[1])[0]
|
||||
if greyed:
|
||||
return report("F5", REPRO,
|
||||
f"{greyed}/{m['n']} enabled inputs fill #f4f4f4 (most common: {top[0]})")
|
||||
return report("F5", FIXED, f"{m['n']} enabled inputs, none grey-filled (most common: {top[0]})")
|
||||
|
||||
|
||||
# ── F6 ────────────────────────────────────────────────────────────────────────
|
||||
def f6(page, base, tok):
|
||||
"""The creator is one very tall form: the review counted 11 cards and ~4,700px
|
||||
with jump links standing in for structure. Threshold: no sectioning control
|
||||
(tabs/steps) and every card in the document flow at once."""
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440)
|
||||
page.goto(base + "/wp-creation-index.html?project=projA",
|
||||
wait_for="!!document.querySelector('.card, .section')")
|
||||
time.sleep(1.0)
|
||||
got = page.eval("""(function(){
|
||||
var h=document.documentElement.scrollHeight;
|
||||
var cards=document.querySelectorAll('.card,.wp-card,section.card').length;
|
||||
var vis=0;
|
||||
document.querySelectorAll('.card,.wp-card,section.card').forEach(function(e){
|
||||
var s=getComputedStyle(e);
|
||||
if(s.display!=='none'&&s.visibility!=='hidden') vis++;});
|
||||
var tabs=document.querySelectorAll('[role=tab],.tab,.section-tab').length;
|
||||
var jump=document.querySelectorAll('a[href^="#"]').length;
|
||||
return JSON.stringify({h:h,cards:cards,vis:vis,tabs:tabs,jump:jump});})()""")
|
||||
m = json.loads(got)
|
||||
if m["vis"] > 1 and not m["tabs"] and m["h"] > 3000:
|
||||
return report("F6", REPRO,
|
||||
f"{m['vis']} cards in one {m['h']}px scroll, {m['tabs']} tabs, "
|
||||
f"{m['jump']} jump link(s)")
|
||||
if m["tabs"]:
|
||||
return report("F6", FIXED,
|
||||
f"{m['tabs']} sectioning control(s), {m['vis']} card(s) visible, {m['h']}px")
|
||||
return report("F6", UNKNOWN,
|
||||
f"{m['vis']} card(s), {m['h']}px, {m['tabs']} tabs - below the 3000px threshold")
|
||||
|
||||
|
||||
PROBES = {"F1": f1, "F2": f2, "F3": f3, "F4": f4, "F5": f5, "F6": f6}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("items", nargs="*", default=[], help="subset, e.g. F2 F5")
|
||||
ap.add_argument("--base-url", default="")
|
||||
args = ap.parse_args()
|
||||
|
||||
wanted = [i.upper() for i in args.items] or list(PROBES)
|
||||
bad = [i for i in wanted if i not in PROBES]
|
||||
if bad:
|
||||
print(f"unknown item(s): {', '.join(bad)}; known: {', '.join(PROBES)}")
|
||||
return 2
|
||||
if not cdp.find_browser():
|
||||
print("no headless-capable browser found (set WP_BROWSER)")
|
||||
return 2
|
||||
|
||||
tmpdir = tempfile.mkdtemp(prefix="wpsuite-f-items-")
|
||||
db_path = os.path.join(tmpdir, "f.db")
|
||||
proc = browser = None
|
||||
try:
|
||||
tok = seed(db_path)
|
||||
add_long_project(db_path)
|
||||
if args.base_url:
|
||||
base = args.base_url.rstrip("/")
|
||||
else:
|
||||
port = cdp.free_port()
|
||||
proc = start_server(port, db_path)
|
||||
base = f"http://127.0.0.1:{port}"
|
||||
if not proc:
|
||||
print("the server would not start")
|
||||
return 2
|
||||
|
||||
print(f"\n probing {', '.join(wanted)}\n")
|
||||
browser = cdp.Browser()
|
||||
page = browser.page()
|
||||
for item in wanted:
|
||||
try:
|
||||
PROBES[item](page, base, tok)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
report(item, UNKNOWN, f"probe raised: {str(exc)[:120]}")
|
||||
|
||||
r = sum(1 for _, v, _ in _RESULTS if v == REPRO)
|
||||
f = sum(1 for _, v, _ in _RESULTS if v == FIXED)
|
||||
u = sum(1 for _, v, _ in _RESULTS if v == UNKNOWN)
|
||||
print(f"\n {_c(str(r) + ' reproduce', '31')}, {_c(str(f) + ' fixed', '32')}"
|
||||
+ (f", {_c(str(u) + ' inconclusive', '33')}" if u else "") + "\n")
|
||||
return 0
|
||||
finally:
|
||||
if browser:
|
||||
browser.close()
|
||||
if proc:
|
||||
proc.kill()
|
||||
proc.wait(timeout=10)
|
||||
try:
|
||||
from server.db import engine
|
||||
engine.dispose()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||