T0.2 - baseline captured; all six rendering defects confirmed present
Runs the app from a clean database, captures the before images, and records which of F1-F6 actually still reproduce. All six do. Rather than eyeball screenshots, each defect is measured in a browser by tests/f_items.py, which reports REPRODUCES / FIXED / INCONCLUSIVE and never a silent pass. That makes it both the wave 0 record and the wave 1-3 regression check: an item is done when its probe flips to FIXED. F1 hero says "Job A", app bar still says "Select a project", no reload F2 "Sign out" spans x382-432 in a 390px viewport - cut in half, 3 rows F3 chrome paints over the logo by 106x32px; .header-left collapses to 0 F4 comments drawer overlaps the header by 380x91px in the standalone creator F5 5 of 5 ENABLED wizard inputs compute #f4f4f4 on #e0e0e0 F6 11 cards in one 5,017px scroll, 0 tabs (review said ~4,700px; it grew) Three probes needed care to avoid reporting a false pass, and the traps are worth knowing before anyone verifies a fix: F1 disappears if localStorage is primed first, because then both sources of truth agree. The probe clears it and drives the real picker. F3 needs a long project name that is long IN THE DATABASE - any page reached with ?project= re-pulls it and overwrites a locally-faked one. It also cannot be measured by comparing .header-left to the chrome: under the long name .header-left (flex:1, min-width:0) collapses to clientWidth 0, so that comparison reports a tidy zero gap while the chrome paints across the logo. It measures against .logo, which is flex-shrink:0. My first two attempts at this probe both reported FIXED for those reasons; the screenshot did not. F5 must ignore genuinely disabled inputs or a fix looks done while real fields stay grey. 14 screenshots, not the 12 the plan asks for, because there are 7 pages (file-map D1). Capture also measures horizontal overflow, which is how BL-001 was found. Tooling: cdp.py gains viewport() and screenshot() - it could do neither, and T0.2 requires 390px and 1440px images. 390px sets the mobile flag rather than just narrowing the window, since every page declares width=device-width and Chrome otherwise lays out at 980px and no media query under test fires. Both new scripts reuse browser_check.py's seed() and start_server() instead of growing a second fixture. Existing browser_check still passes 71/71. No application code changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
212
tests/baseline_shots.py
Normal file
212
tests/baseline_shots.py
Normal file
@@ -0,0 +1,212 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Capture the suite's pages at both reference widths, for before/after comparison.
|
||||
|
||||
CLAUDE.md asks every frontend task to exercise the affected flow at 390px and at
|
||||
1440px and to put before and after screenshots in the PR. Doing that by hand 57
|
||||
times is how it stops getting done, so it is a script.
|
||||
|
||||
python tests/baseline_shots.py # -> docs/reference/baseline/
|
||||
python tests/baseline_shots.py --out /tmp/after # the "after" half of a diff
|
||||
python tests/baseline_shots.py --pages creator,field
|
||||
python tests/baseline_shots.py --widths 390,768,1024,1440
|
||||
|
||||
Self-contained, like tests/browser_check.py, whose seed() and start_server() it
|
||||
reuses rather than growing a second fixture: throwaway SQLite, its own uvicorn,
|
||||
headless Edge or Chrome over CDP, everything torn down afterwards. Your real
|
||||
database is never touched.
|
||||
|
||||
390px is emulated with the mobile flag set, not merely as a narrow desktop window.
|
||||
Every page in html/ declares width=device-width, so this is the layout a field
|
||||
tablet actually gets; without the flag Chrome lays out at 980px and the media
|
||||
queries under test never fire. The script asserts the width it asked for is the
|
||||
width the page saw, because that failure is otherwise invisible in a PNG.
|
||||
|
||||
Exit codes: 0 all captured · 1 one or more pages failed · 2 could not run.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import cdp # noqa: E402
|
||||
from browser_check import seed, start_server # noqa: E402
|
||||
|
||||
# Page copy contains em dashes and other non-cp1252 characters, and the default
|
||||
# Windows console encoding raises UnicodeEncodeError on them mid-run.
|
||||
try:
|
||||
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||
except (AttributeError, ValueError): # pragma: no cover
|
||||
pass
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
DEFAULT_OUT = os.path.join(REPO, "docs", "reference", "baseline")
|
||||
|
||||
# The active project is seeded as Job A. project-data.js keeps the id and a
|
||||
# denormalised copy under these two keys; both are set, because a page that reads
|
||||
# only the object would otherwise render its empty state.
|
||||
ACTIVE_ID = "projA"
|
||||
ACTIVE_OBJ = {"id": "projA", "name": "Job A", "number": "A-1", "client": "Internal QA"}
|
||||
|
||||
# name, file, user whose session to use, JS that means "this page has its data".
|
||||
# login is visited signed OUT — it is the one page whose real state is no session.
|
||||
PAGES = [
|
||||
("login", "login.html", None, None),
|
||||
("launcher", "index.html", "root", "!!document.querySelector('body')"),
|
||||
("sop", "work-package-suite.html", "root", "!!document.querySelector('.header-left, header')"),
|
||||
("creator", "wp-creation-index.html", "root", "!!document.querySelector('#wp_number, .field')"),
|
||||
("admin", "admin.html", "root", "!!document.querySelector('main, .card')"),
|
||||
("field", "field.html", "root", "!!document.querySelector('body')"),
|
||||
("users", "users.html", "root",
|
||||
"!!document.querySelector('#users-table table, #users-table .note:not(:empty)')"),
|
||||
]
|
||||
|
||||
_OK, _BAD, _OVERFLOW = [], [], []
|
||||
|
||||
|
||||
def _c(s, code):
|
||||
return f"\033[{code}m{s}\033[0m" if sys.stdout.isatty() else s
|
||||
|
||||
|
||||
def capture(page, base, tok, name, filename, user, wait_for, widths, out, label):
|
||||
"""Shoot one page at every width. Returns True if all of them landed."""
|
||||
ok = True
|
||||
page.clear_cookies()
|
||||
if user:
|
||||
page.set_cookie("wp_session", tok[user])
|
||||
|
||||
# localStorage is per-origin, so prime it once on this origin before the real
|
||||
# navigation. Signed-out login.html is left alone: giving it an active project
|
||||
# would be staging a state that page never has.
|
||||
if user:
|
||||
page.goto(base + "/index.html")
|
||||
page.eval(
|
||||
f"localStorage.setItem('wp_active_project', {ACTIVE_ID!r});"
|
||||
f"localStorage.setItem('wp_active_project_obj', {json.dumps(ACTIVE_OBJ)!r});"
|
||||
"true")
|
||||
|
||||
for w in widths:
|
||||
page.viewport(w, 900, mobile=(w <= 500))
|
||||
page.goto(base + "/" + filename, wait_for=wait_for)
|
||||
time.sleep(0.5) # webfonts and late-injected chrome
|
||||
|
||||
# Ask the page how wide it actually ended up. A page whose content will not
|
||||
# fit forces the initial containing block wider than the device, so innerWidth
|
||||
# comes back above what was requested and everything in the shot is at the
|
||||
# wrong scale. That is a finding about the page, not a failure of the capture,
|
||||
# so it is measured and reported and the screenshot is still taken.
|
||||
m = page.eval(
|
||||
"JSON.stringify({inner: window.innerWidth,"
|
||||
" scroll: document.documentElement.scrollWidth,"
|
||||
" client: document.documentElement.clientWidth})")
|
||||
m = json.loads(m)
|
||||
overflow = m["inner"] != w or m["scroll"] > m["client"]
|
||||
if overflow:
|
||||
_OVERFLOW.append(
|
||||
f"{name}@{w}: laid out {m['inner']}px, content {m['scroll']}px "
|
||||
f"in a {m['client']}px viewport")
|
||||
|
||||
suffix = f"-{label}" if label else ""
|
||||
path = os.path.join(out, f"{name}-{w}{suffix}.png")
|
||||
try:
|
||||
page.screenshot(path)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
_BAD.append(f"{name}@{w}")
|
||||
print(" " + _c("FAIL", "31") + f" {name} {w}px — {exc}")
|
||||
ok = False
|
||||
continue
|
||||
|
||||
errs = page.js_errors()
|
||||
size = os.path.getsize(path)
|
||||
_OK.append(f"{name}@{w}")
|
||||
flags = []
|
||||
if overflow:
|
||||
flags.append(f"overflows: {m['scroll']}px of content")
|
||||
if errs:
|
||||
flags.append(f"{len(errs)} JS error(s)")
|
||||
note = " " + " · ".join(flags) if flags else ""
|
||||
print(" " + _c("OK", "32") + f" {name:9s} {w:>4}px {size:>7,}b "
|
||||
f"{os.path.basename(path)}{_c(note, '33')}")
|
||||
for e in errs[:3]:
|
||||
print(f" {_c('js:', '33')} {e[:110]}")
|
||||
return ok
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--out", default=DEFAULT_OUT, help="directory for the PNGs")
|
||||
ap.add_argument("--widths", default="390,1440", help="comma-separated CSS widths")
|
||||
ap.add_argument("--pages", default="", help="comma-separated subset of page names")
|
||||
ap.add_argument("--label", default="", help="suffix, e.g. --label after")
|
||||
ap.add_argument("--base-url", default="", help="use a server that is already up")
|
||||
args = ap.parse_args()
|
||||
|
||||
widths = [int(w) for w in args.widths.split(",") if w.strip()]
|
||||
wanted = {p.strip() for p in args.pages.split(",") if p.strip()}
|
||||
pages = [p for p in PAGES if not wanted or p[0] in wanted]
|
||||
if wanted - {p[0] for p in PAGES}:
|
||||
print(f"unknown page(s): {', '.join(sorted(wanted - {p[0] for p in PAGES}))}")
|
||||
print(f"known: {', '.join(p[0] for p in PAGES)}")
|
||||
return 2
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
|
||||
if not cdp.find_browser():
|
||||
print("no headless-capable browser found (set WP_BROWSER)")
|
||||
return 2
|
||||
|
||||
tmpdir = tempfile.mkdtemp(prefix="wpsuite-baseline-")
|
||||
db_path = os.path.join(tmpdir, "baseline.db")
|
||||
proc = browser = None
|
||||
try:
|
||||
tok = seed(db_path)
|
||||
if args.base_url:
|
||||
base = args.base_url.rstrip("/")
|
||||
else:
|
||||
port = cdp.free_port()
|
||||
proc = start_server(port, db_path)
|
||||
base = f"http://127.0.0.1:{port}"
|
||||
if not proc:
|
||||
print("the server would not start")
|
||||
return 2
|
||||
|
||||
print(f"\n {len(pages)} page(s) x {len(widths)} width(s) -> {args.out}\n")
|
||||
browser = cdp.Browser()
|
||||
page = browser.page()
|
||||
for name, filename, user, wait_for in pages:
|
||||
capture(page, base, tok, name, filename, user, wait_for,
|
||||
widths, args.out, args.label)
|
||||
|
||||
print(f"\n {_c(str(len(_OK)) + ' captured', '32')}"
|
||||
+ (f", {_c(str(len(_BAD)) + ' failed', '31')}" if _BAD else ""))
|
||||
if _OVERFLOW:
|
||||
print(f"\n {_c('horizontal overflow', '33')} "
|
||||
f"({len(_OVERFLOW)} of {len(_OK)} shots) — content wider than the "
|
||||
f"viewport it was asked for:")
|
||||
for line in _OVERFLOW:
|
||||
print(f" {line}")
|
||||
print()
|
||||
return 1 if _BAD else 0
|
||||
finally:
|
||||
if browser:
|
||||
browser.close()
|
||||
if proc:
|
||||
proc.kill()
|
||||
proc.wait(timeout=10)
|
||||
# seed() built an engine in this process too; drop it before deleting the
|
||||
# file or Windows keeps the handle open. Same reason as browser_check.
|
||||
try:
|
||||
from server.db import engine
|
||||
engine.dispose()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user