Click a label to see where it sits on a work's back and which other works in the SKKG collection carry it. What the filters and badges mean →
The backs of the works with the labels located on them: click a work for its front, its exploded back and its provenance timeline. How to read a work →
Colours: label register label photo data sheet model (unverified) label detector same label as on another work
What this is
Labels of dealers, auction houses, exhibitions and transport firms on the backs of artworks are traces of their history. This site shows where such labels, stamps and marks sit on the backs of works in the collection of the Stiftung für Kunst, Kultur und Geschichte (SKKG), Winterthur, and which works carry the same label. It was made at GLAMhack 2026 for challenge 304 «Provenance Pursuit».
For visitors it is a way to browse the backs and the histories they document. For the team it is also a workbench: boxes are checked and corrected here, and labels are recorded for SKKG's label register. Every picture is cut from SKKG's photographs by SKKG's IIIF image server at the moment you look at it; the site itself holds only boxes (x, y, width and height in pixels of the original image) and text.
The tabs
Labels Open the tab →
- Shows
- Every label, stamp and mark placed on a back ( in all), each as a picture cut from the photograph of the back, with its work and its badges. They come from four sources: Label register: entries of SKKG's label register, placed by matching the register's picture with the back; Label photos: detail photographs SKKG took of labels, placed the same way; Data sheets: of the marks described in SKKG's provenance data sheets, placed by their text, by a template or by a model's proposal, one box per mark (where several marks of a data sheet landed on the same box, the one with the stronger evidence or the better-fitting text keeps it; marks that cannot be told apart share it, «undecided»); Detected: labels and stamps that the label detector found and no record documents.
- How to use
- Search looks in the label's title (for register entries, the issuer), its text or description, event, position and type, and in the work's inventory number, title, object type and provenance actors.
- Source: All, Label register, Label photos, Data sheets, Detected.
- found on other works: only labels that were found again on another back.
- Show, «Reliable only»: only the reliable boxes: matches with the register's picture or SKKG's detail photo, data sheet text read where the data sheet places it (also after an image filter), and templates of the same issuer.
- Show, «With unverified leads» (the default is «All located») adds boxes: the proposals of the vision-language model and the detector finds without readable text. The «unverified boxes» checkbox in a work's dialog is the same switch.
- Data sheet facets (in «Filters», after source, «Show» and «found on other works»): issuer, owner, type, kind of issuer, event, date, place, country, language, technique, support, position, side, shape, edge, colours and writing medium of the labels that a provenance data sheet describes, as the prov-parser read the descriptions (and register entries that show such a mark). Values of one facet add up (or), facets narrow down (and), together with the search and the other filters; the number beside a value says how many labels it would leave. Values are in German as in the data sheets, with English glosses; names in their most frequent spelling, the spellings of one actor merged by SKKG's list of actors; long lists show the most frequent first, with «Show all» and a find field; «andere» gathers values of fewer than five marks. LLM: filled by a language model, verify in the data sheet; ?: uncertain (inferred while coding, or read from the photo or the provenance chain), left out with «only certain values». The address keeps all the filters, so a link shows the same selection.
- The filters combine, and «Filters» folds them away (on a phone they start folded); the count says how many labels match, and «Show more» loads the next 96. Labels found on the most works come first, then the reliable ones. Click a card for the label dialog.
- Watch for
- By default of the labels are shown. Every box that is not reliable carries to check or, if it is an unverified lead, unverified. An amber «Data sheet» badge is a box the model proposed: unverified. For detected labels, issuer and kind are guesses from the text read by OCR, and OCR misspells names.
Works Open the tab →
- Shows
- The backs of the works with their boxes drawn on them (the back with the most labels when a work has several photos of its back), the number of labels located and, in green, «shares a label with another work».
- How to use
- Search by inventory number, title, object type, provenance actor, exhibition, acquisition or the text of a label. «only works with located labels» (on by default: works) and «only works that share a label» () narrow the list. Works with the most labels come first.
- Data sheet facets: the same groups, in «Filters», for the works with a provenance data sheet. A work counts for a value when any mark its data sheet describes has it, also a mark on the front or one that was not placed on the back. Combined with the search and the switches; the address keeps them.
- Click a work, for example Inv. 4954. At the top of its dialog: The back, exploded beside the Provenance timeline. A work without located labels shows the timeline only.
- Explode: drag the slider. The back shrinks to half its size and every label lifts onto a ring around it, without overlapping the others; dashed holes and thin lines show where each one sat. Each label is a button with its own cut-out. When there are several photos of the back, «Back 1», «Back 2» … switch between them. A label that several records place on the same spot is one piece: the register, a label photo and a data sheet that describe the same label, or several data sheet entries placed on one box (records whose boxes overlap by half or more). The piece shows the box of its most reliable record; its panel lists the others.
- Provenance timeline: the stations of the work's provenance as SKKG's records give them, in order (the artist, owners, estates, dealers, auction houses, museums), its exhibitions and its acquisition by SKKG. Under an event, «●●○ 2 labels» says how many labels document it and how strong the best link is.
- Click an event: its labels light up and the others fade, the back explodes and switches to the photo that holds them, and the panel under the back lists them with the basis of each link. Click a label: a small card beside it leads to its «Label details →»; its events light up in the timeline, each with its basis; and the panel under the back names the other records on the same box and offers «Label details», «✎ Record in the entry form», «Whole back, zoomable» and «Report a problem with this label or its events». A second click deselects. Back from a label's dialog (or through its «Work» link), the work opens on its back, exploded, with that label picked.
- A label without a box? «+ Draw a missing label», under the exploded back and beside each back photo, opens that photo zoomable with the boxes already on it: draw the box around the label, then «Record this label →» opens the entry form for it as a new label. New labels drawn in this browser are listed next to the button and shown as dashed green boxes on the back photo until they are merged into the data.
- Strong, medium, weak. A link is strong (●●●) when two features agree: the issuer and the lot, date or year of a sale, or the venue and year of an exhibition. It is medium (●●○) for one distinctive feature: the issuer or venue alone, a city and a year next to an exhibition word, an owner's full name in the register or a data sheet, or the same label on another work that shares one exhibition, seller or actor. It is weak (●○○) when it is ambiguous: several events fit, the issuer appears with another year, a name was read only approximately, or a city and a year stand alone. The basis also says where the text was read: register, data sheet, OCR of the label photo or OCR of the detected label.
- The boxes: below the exploded back come the facts (object type, collection page, acquisition, provenance actors, exhibitions) and «Labels it shares with other works». Under them, folded, «Boxes on the back» shows every photo of the back with all its boxes, one per spot: click a box to open the entry form for that label, or «Open zoomable with all boxes». At the end, every located label as a card, one per spot; «+2 records» on a card says that more records place a label on the same box.
- Watch for
- works have a timeline, with events in all; of them are documented by a label ( strongly). The timeline holds only what SKKG's records hold, so a label without an event may document a station or an exhibition the records lack. The order is reconstructed: an undated station stays after the dated one before it, and exhibitions are placed by their year. «+1 with an unverified box» under an event means that a hidden box documents it too. A link is evidence to weigh, not proof. Owners, heirs and sellers are named as SKKG's public records name them.
Found again Open the tab →
- Shows
- located labels that were found again on the backs of other works, with
hits. Each row starts with the label and its work; after the arrow come the backs where the
same label was found: the cut-out there, the number of points that agree between the two pictures (matching points), and the
evidence from the records of both works:
- «Exhibition: …»: both works were lent to this exhibition;
- «Same seller: …»: SKKG bought both from the same seller;
- «Provenance: …»: the other work's provenance names the label's issuer or shares a dealer or auction house;
- «no evidence in the records yet».
- How to use
- Filter by evidence: All hits, Shared exhibition, Same seller, Provenance, No evidence yet. Labels with the most hits come first. Click the label for its dialog, or a hit for the other work's dialog, where the hit appears as a green dashed piece. Each hit has «Report», for a problem with this match.
- Watch for
- A hit is the same label design, confirmed by image matching: a lead, not proof. It can be an identical label on related works, one issuer's standard label used for decades, or one of SKKG's own loan strips with a crate number, which a shared exhibition «backs» although it documents the loan rather than an earlier owner. One exhibition, «Sesam öffne dich!» (Bern and Martigny, 2014/15), showed about 70 of the works, so many hits share it. Only the labels from the register and from SKKG's detail photos that were located reliably were searched.
The label dialog Open an example →
- Shows
- One label: its cut-out and, where there is one, SKKG's own detail photo; the facts: work (with links to the work and to SKKG's collection), source, «Located by» (image match with the number of matching points, the most similar window, text read by OCR, a template from another work, the vision-language model, or the detector's score), position, type, description or text read, issuer guess, and the provenance events it is linked to. «Same spot» names the other records that place a label on the same box, which the work's back shows once. Below: where it sits on the back, the same label on other works, and the other marks on the same work.
- How to use
- Click any card. «✎ Record this label» opens the entry form, and so does
a click on the box on the back. «+ Draw a missing label on this back», under the back, is for a label on the
same back that has no box yet: draw it, zoomable, and it opens in the entry form as a new label. The cut-out and «Whole back, zoomable» open the box viewer;
«Full resolution» opens the cut-out itself; «Report a problem» opens the
feedback form. Close with ✕, Esc or a click beside the dialog. Every dialog
has its own address:
#label=and the label's key (for example#label=register:52), or#work=and SKKG's object ID, the number in the work's collection address. - Watch for
- to check or unverified after «Located by» marks a box that is not reliable. «Kind (guess)», «Issuer (guess)» and «Text read (OCR)» of a detected label are machine readings: expect misspellings; addresses are masked («[address]»).
The other pages
Entry form Open an example →
- Shows
- A form for columns F to L of SKKG's label register, one label at a time: the label's picture with «Ways of seeing»; «What the data already say» (the whole back with the box, the source, how the box was found, the same label on other works); the fields Label (F), Position (G), Attribution (H to J) and Further details (K, L); and a preview of the register row.
- How to use
- Open it from a box (click it), from «✎ Record this label», or from a box in the box viewer.
Its address is
form.html?label=and the label's key; without a key it is an empty form. A label drawn with «+ Draw a missing label» opens here as a new label (keynew:, the back photo and the box); it is kept at once, with its box confirmed, and nothing is prefilled but the position and side. - Prefill: what the data hold is filled in and tagged. A pale blue field tagged «Register», «Data sheet» or «Box» comes from the data. A pale amber field tagged «Suggestion», «OCR» or «Image search» is derived: shape, colour and lettering from the description, the position from where the box sits, the exhibition or seller shared with the same label on other works, an issuer guessed from the text read. Check these. Once you change a value the tag reads «edited ↺»; ↺ restores the value from the data. «From the data» under each section quotes the source.
- Ways of seeing: filters that help to read faint pencil, blue crayon and stamps, computed in your browser. Enhance: Photo, Local contrast, Flat field, High-pass, Decorrelation stretch, Threshold. One colour channel: Red, Green, Blue, Blue ink, Red ink. «Invert» combines with any of them. Filters with a parameter get a slider (double-click resets it); ⤢ enlarges the picture; each way explains itself in one line. Where SKKG has a detail photo of the label, «Picture» switches between the back photo and that photo.
- Confirm or redraw the box: «✓ Confirm box» when it sits on the label. «✎ Redraw box», or a click on the small back, opens the whole back zoomable on the box (scroll or pinch to zoom; Space + drag or «Move picture» moves it): fit the box by its corners and edges, or drag a new one (coloured: the box from the data, green: yours); «✓ The box fits» confirms it as it is. It is kept in pixels of the original image and counts as confirmed; «Restore the box from the data» undoes it. The picture, the back and the viewer link follow the new box.
- Copy row copies the register row, tab-separated, to paste into column F of the Excel register. «Reset» restores the values from the data.
- CSV: every change is saved in this browser, per label. Enter your initials, then «Download CSV»
(
etiketten_erfassung_<initials>_<date>.csv: label key, inventory number, the box and whether it was confirmed or redrawn, columns F to L and the single fields, initials, time) and put the file into the team folder, where the entries are merged into the workbook. «Load CSV …» takes over the entries of such a file; for the same label the newer entry wins.
- Open it from a box (click it), from «✎ Record this label», or from a box in the box viewer.
Its address is
- Why German
- SKKG's label register is German, so what the form writes into it stays German: the values of its lists and buttons (shown with English glosses, such as «Zierrahmen · frame»), the column headings F to L, the register's conventions («o. D.» = no date, «(Lot …)») and the comment it prefills in column L on how the box was found. A copied row can go into the register as it is. Everything around it is English.
- Watch for
- Entries stay in this browser only, and a private window may not keep them (the form warns): download the CSV before you switch browser or computer. Suggestions are guesses. The position grid is captioned «seen from the front», but the positions prefilled from the data and from the box are as seen on the back photo. The filters store and upload nothing, and the chosen way is not remembered. Comments already in the register's column L are not published; the form only says that there is one, so keep it when you paste.
Review page Open it →
- Shows
- The vision-language model's box proposals for data sheet marks that text recognition could not find, one at a time: the proposed box (purple) with its surroundings, the whole back with the box and the area the data sheet names (white, dashed), and the data sheet's description.
- How to use
- Decide with ✓ Correct (Y), ✗ Wrong (N) or ? Unsure (U). If the mark is visible
but the box is wrong or too loose, ✎ Redraw box (R): the whole back opens, zoomable, with the box ready to
fit by its corners and edges, or drag the right one (green). The picture of the proposal zooms too: + and − in
its corner, a double-click, a pinch or Ctrl/⌘ + scroll; drag to move it. ←
→ move between proposals; after a decision the next open one follows. Filter by status (not yet reviewed,
all, correct, redrawn, wrong, unsure), by type (labels, stamps, inscriptions, other) or with the search. Enter your
initials, then «Download my decisions (CSV)» (
model_box_review_<initials>_<date>.csv) and put the file into the team folder; «Load decisions …» reads such a file back. - Watch for
- The model answers with a box even when the mark is not visible, so a neat box can still be wrong. Decisions stay in this browser until you download them. When the files are merged, the latest decision per mark counts: confirmed boxes count as located, rejected ones fall back to the area the data sheet names. Confirmed boxes merged so far: .
Search by photo Open it →
- Shows
- «Have I seen this label before?»: the 20 backs, out of , that look most like your photo of a label, stamp or inscription. Each comes with its best-matching window, cut out by the IIIF server, its rank and score, «Show on the back» (box viewer) and «SKKG record».
- How to use
- «📷 Take photo» or «Choose image…», drag a box around the label if you want to search only that part («🔍 Zoom in
to crop» to draw it exactly), then
«Search». The first search downloads the image model (about 174 MB, then cached) and the search index (about
8 MB); after that a search takes a few seconds. To help check the hits: ✓ Same, ✗ Not it, ?, and «Redraw box» to
draw the exact box on that back, zoomable (also when the label sits elsewhere on it). With your initials and a note on what you
photographed, «Download my decisions (CSV)» (
label_search_review_<initials>_<date>.csv) for the team folder. - Watch for
- Your photo never leaves your browser: the model runs on your device. Only the coarse search runs there: for test photos it puts the right back first in of the searches, while the full pipeline, which re-checks the candidates by image matching, reaches . Treat the hits as candidates to look at, not as identifications. On a phone, make the first search on Wi-Fi.
Box viewer Open an example →
- Shows
- A whole image from an IIIF image server, zoomable, with one or more boxes on it: «Whole image» and «Zoom to box», and links to the whole image and to each box as IIIF requests, to the collection page and to the entry form.
- How to use
- Every «Whole back, zoomable» link on this site and every «view» link in the workbook opens it. Scroll or pinch to
zoom, drag to move. Its address takes these parameters:
iiif- the IIIF image service (required); its
info.jsonor any image address of it works too xywh=x,y,w,h- a box in pixels of the original image; for several boxes, repeat it or
separate them with
; area=x,y,w,h- an approximate area, always dashed
colour=ff3b30- the colour of the boxes, as hex without
#(colorworks too) style=dashed- dashed boxes, for an approximate position
title,note- a heading and a line of text above the image
zoom=1- open zoomed in on the boxes
static=1- one plain picture instead of deep zoom
label=key- one per box, in the same order: a click on the box opens the entry form for that label
page=address- adds a «Collection page» link
- Watch for
- Boxes are in pixels of the original image, not of the picture on your screen; the image service's
info.jsongives the full size. The image comes straight from the IIIF server; if deep zoom cannot start, the viewer shows one plain picture instead.
What SIK-ISEA adds Open it →
- Shows
- The records of SKKG works in SIK-ISEA's research portal, compared with SKKG's knowledge graph, as counts and links only: how many SIK-ISEA records were linked to an SKKG object, how many exhibition venues and institutional ownership stations SIK-ISEA records that the knowledge graph lacks, how many of these a label on the back supports, and the works with such a label, each with links to the SIK-ISEA record, to SKKG's collection and to the label's box in the box viewer.
- How to use
- Start with the counts, then follow a work's links. The evidence column says how a label supports a record: strong (issuer or town and year, or a lot or catalogue number, on the label), issuer only, the same touring show at another venue, or place only.
- Watch for
- SIK-ISEA's content may not be republished without SIK-ISEA's consent, so the page shows only counts, links and our own reading of the labels; the details are in SIK-ISEA's records. «Place only» is a weak lead. Private owners are never named.
Workbook Download (Excel) →
- Shows
- All results in one Excel file: a guide and the key numbers on «Overview», then the sheets Register labels, Label photos, Discovery (labels found again), What SIK-ISEA adds, SIK-ISEA links, Search evaluation, Data sheet marks, Detected labels, Works, Provenance candidates, and Issuers and actors.
- How to use
- Every box has two links: «region» opens the cut-out on the IIIF server, «view» the whole image with the box in the box viewer. Sort and filter as in any spreadsheet.
- Watch for
- This is the public copy: names appear as SKKG's public records and the labels give them, addresses are masked («[address]»), and SIK-ISEA appears only as links and counts. Apart from the label register's own pictures it holds no images. It is a snapshot from the build date in the footer.
Feedback Send feedback
- Shows
- A «Feedback» button at the bottom right of every page (in an open dialog it moves into the dialog), and «Report a problem …» or «Report» links next to boxes, labels, timeline events and matches.
- How to use
- Choose what it is about (a box is wrong or missing, a text or data value is wrong, something doesn't work, an idea or wish, other), say what you saw and what you expected, look over the lines sent along (page, work, label key, box; edit or delete them) and press «Send». You need no GitHub account. If you have one, «Open on GitHub instead» opens the same issue, prefilled.
- Watch for
- Your message becomes a public issue in the site's GitHub repository when the site's GitHub Action next picks it up; GitHub starts that Action on a timer that can lag by hours. Names from the records are fine; please don't write addresses or your own contact details. Your text goes in as plain text, together with the lines under «Sent along» (page, the work's SKKG title, label key, box, browser), never a label's text.
For the team: from your browser to the workbook
Three pages keep your work in your browser: the entry form, the review page and the search by photo. Nobody else sees it until you download your file:
- Use the same initials on all three pages.
- When you are done, and before you switch browser or computer, download the CSV.
- Put the file into the team folder. The files are merged into the workbook and this site when they are next rebuilt; for the same label the newest entry or decision counts.
Every label has a stable key, used in the addresses, the CSV files and the feedback: register:<row>,
photo:<object ID>:<IIIF ID>, sheet:<hash>,
detected:<IIIF ID>:<x,y,w,h>, and for a box found by the search
box:<IIIF ID>:<x,y,w,h>.
Colours, badges and «to check»
Badges on the cards
- Label register
- placed from a picture in SKKG's label register
- Label photo
- placed from SKKG's detail photo of the label
- Data sheet
- a mark described in a provenance data sheet, placed by its text or by a template
- Data sheet
- the same, but the box was proposed by the vision-language model: unverified
- Detected label
- found by the label detector; no record documents it
- on 2 other works
- found again on the backs of other works (on a work's card: «shares a label with another work»)
- to check
- not reliable: an uncertain image match, a register picture placed only roughly (the most similar window), data sheet text that matched weakly or was found elsewhere than the data sheet says, a similar label used as a template, or a detector find with text
- unverified
- a lead: a model proposal, or a detector find without text
Boxes on a back and pieces of the exploded back
- solid, in the colour of the badge: label register, label photo, data sheet, detected label with text
- dashed: unverified, a model proposal or a detected label without text; the Labels tab and the dialogs show them only with «With unverified leads», the cards of the Works tab always draw them
- dashed green: the same label as on another work, found here by the search
- on a lifted piece: the number of timeline events the label is linked to
Timeline and links
- a station of the provenance: the artist, an owner, an estate, a dealer, an auction house, a museum, SKKG
- an exhibition the work was lent to
- the acquisition by SKKG
- an undated event
- ●●●
- strong link: two features agree
- ●●○
- medium link: one distinctive feature
- ●○○
- weak link: ambiguous, a lead
Select an event and its labels get a frame in these styles while the others fade; select a label and its events are tinted the same way (darker for strong, a dashed outline for weak).
On the other pages
- Entry form: pale blue fields come from the data, pale amber fields are suggestions to check, «edited ↺» marks your changes. The box keeps the colour of its source (dashed if unverified); a box you drew is green. In the redraw dialog, red is the box from the data and green is yours.
- Review page: purple is the model's box, the white dashed frame the area the data sheet names, green your redrawn box. The progress bar counts correct (green), redrawn (blue), wrong (red) and unsure (amber).
- Search by photo: a green frame marks «Same», a dashed amber frame «?», a faded card «Not it».
- Box viewer: the box has the colour its link gives, usually the colour of its source; dashed means an approximate position or area.
How it was found, in short
- Image index. All photographs of backs were cut into overlapping windows and described with DINOv2, an image model, so that a photo of a label can be searched like a picture search.
- Image matching. SuperPoint, LightGlue and MAGSAC check the most similar windows: the same physical label has dozens to hundreds of points that agree geometrically. This places the label register's pictures and SKKG's detail photos on the backs, in pixels of the original image; of the test photos find their back at rank 1.
- Reading the text. The marks described in SKKG's provenance data sheets were looked for on the backs by their text (OCR), some only after an image filter; labels already located served as templates for a few more.
- A vision-language model (Qwen3-VL, running locally) proposed boxes for the rest: unverified leads.
- A label detector (OWLv2, running locally) searched every back for anything that looks like a label or stamp: finds that no record documents, of them with readable text.
- Found again. Every reliably located label from the register and the detail photos was searched on all other backs: hits, of them backed by the records of both works.
- Timeline. Provenance stations, exhibitions and acquisitions come from SKKG's knowledge graph; a label is linked to an event when its text agrees with the record.
The full story, step by step: How we got here.
Privacy
- Names appear as SKKG's public records and the labels give them: the sources are public, and the names matter for provenance research. Addresses that labels, data sheets or the text recognition transcribe are masked («[address]»). The masking follows patterns and can miss an address: if you see one, report it with «Report a problem» without repeating it.
- In the timeline, owners, heirs, sellers, dealers, auction houses and museums are named as SKKG's public records name them, and provenance actors are shown as SKKG publishes them.
- The comments in column L of SKKG's label register are not published.
- The search by photo keeps your photo in your browser. Entries in the form and decisions on the review and search pages stay in your browser until you download them.
- Feedback becomes a public GitHub issue: names from the records are fine; no addresses and no contact details, please. It travels through a small relay on Google Apps Script that keeps each message for a few hours.
- Besides SKKG's image server, the pages load a few things from elsewhere: the box viewer its zoom library from cdnjs, the entry form its fonts from Google Fonts, the search by photo its image model from Hugging Face and its code library from jsDelivr.
- SIK-ISEA's research portal is only linked and counted.
Data and licences
- Data and images: SKKG Stiftung für Kunst, Kultur und Geschichte, Winterthur (SKKG Sammlung digital, knowledge graph and IIIF image server). Object metadata and photographs of backs CC0; photographs of fronts as SKKG publishes them (mostly CC BY 4.0, some CC0 or ©), credited under each picture with the licence and the rights holder SKKG names; provenance data sheets CC BY 4.0, SKKG provenance research team; label register and object list from the challenge.
- Exhibitions and acquisitions were read from the knowledge graph following skkg-mcp (RISE, University of Basel).
- Data sheet facets: the marks of the provenance data sheets as the prov-parser read them (MIT licence, © 2026 University of Basel; rules for the physical description, a language model for issuer, owner, event, date, place, country and language), matched to the labels by this project (prov_parser/facets.py).
- These pages copy no images: every picture is requested from SKKG's IIIF image server, and boxes are kept as x, y, width and height in pixels of the original image.
- SIK-ISEA's research portal is only linked and counted (What SIK-ISEA adds): its content may not be republished without SIK-ISEA's consent.
- All results as a workbook (Excel) · Source of this site on GitHub · How we got here