Repository navigation
What are the requirements for the scoring system algorithm? #1225
Description
This topic came up in our taskforce, its part of a broader conversation.
Here is a list of requirement that were taking into account when discussing #994
- It MUST be possible to hard wire widgets with
shui:editorandshui:viewer - It MUST be possible to create a custom widget and not touch any scoring graph triples. Using this custom widget can then only be done via
shui:editorandshui:viewer. By adding a scoring shape, it can be picked automatically and theshui:editorandshui:viewerstatement are not needed then. Implementation of a custom widget MUST not require prior RDF knowledge outside of reading the value of the term and updating the term in case its an shui:Editor- Each scoring system function must have one fixed return type (no union types).
- The scoring system MUST use generic programming language features available to most languages
The algorithm most have a reasonably optimal performance- If a widget declared with
shui:editororshui:vieweris not accepted, widget selection MUST continue with the regular scoring, without a separate fallback mechanism. - Closed world assumption must be possible. It can be achieved via the removal of all accept matchers and hard wire all widgets in the shapes.
- Not finding a widget is reported to the interface.
Values.
These are not easily testable. These are things I keep in mind but find hard to make more concrete.
- When a developer implements a custom widget as few as possible RDF knowledge should be required.
- An implementation of the scoring system by following exactly the text, should ideally not result in many unnecessary SHACL validations. Many here is defined as more than 1000 SHACL validations given a form that has 20 fields and the core implementation of the 16 editors.
Activity
[-]What are the requirement for the scoring system algorithm?[/-][+]What are the requirements for the scoring system algorithm?[/+]danielbeeke commentedon Sep 2, 2026
These questions were asked earlier outside of this issue.
about 2
The scoring graph preparation gives each widget without a score shape a 40 (or a defined default score via the global configuration) such that hard wiring works.
about 3
A minimal understand of RDF of reading or writing is required. Indeed, I think it can be very minimal. Its needed for updating one term and reading the term inside the widget code. When using typescript this interface is also shown in types. This minimal understand is also needed to hard wire the widget to the shape.
bergos commentedon Sep 10, 2026
3: I don't think this needs to be explicitly covered by the scoring system. It's more a matter of registering a UI component to an IRI.
9: I think this belongs in the rendering layer. The scoring system has no context about where to display an error or what follow-up actions to take, such as blocking other elements when a single widget can't be found.
bergos commentedon Sep 10, 2026
While working on this, I identified a problem that I thought was worth a separate issue #1248.
smessie commentedon Sep 25, 2026
I find the phrasing of this requirement unclear. Could you rephrase it?
I can read it in several ways. I think you mean something along the lines of: someone writing a custom widget should only have to handle a single RDF term, namely the value the widget displays, or edits and writes back if it is a shui:Editor.
If so, I agree with @bergos that this is not something the scoring system should cover.
Which functions does this cover: all of validation, matcher, accept, score and select? And what do "simple" and "singular" mean here? The score function returns an ordered list of results (widget, score instance, score). Does that meet the requirement, or should it change?
I would propose rephrasing this requirement to remove the ambiguity.
I believe the specification should avoid programming-language-specific concepts and describe the algorithms in abstract terms. How to implement them in a given language should be up to the implementation.
This is a difficult requirement to evaluate: how do we decide whether the performance is "reasonably optimal"?
Beyond that, I think performance is up to the implementation, if that is its focus. The specification should first of all deliver a clear and correct algorithm. The spec already notes that implementations may use an alternative algorithm as long as it results in the same widget selection, which leaves room for such optimisations.
When a scoring graph is used, the scoring system already handles this. If the accept matcher rejects the hard-coded widget, its widget score is filtered out. Any other widget scores (zero, one or several) can still match, and the best-scoring one that is accepted is used.
If none of the matching widget scores is accepted, the scoring system should report this to the application, so that it can indicate that the property shape cannot be rendered.
I see this as one of the strengths of the scoring system, and it needs little additional functionality. So I don't think a separate fallback mechanism is needed here; the scoring system already provides one.
danielbeeke commentedon Sep 27, 2026
@smessie these were requirements I had in mind when we talked about the scoring system.
About 3. I think we align, however there was a previous idea you had where it required the widget creator to always create at least the scoring shapes for hard wiring. This was before adding the preprocessing of the scoring shapes. So this is a requirement that keeps for a nice tidy design.
About 4. I want one return type, not a union. There was a point where we had a return type like:
IRI | Array<{ widget: IRI, scoreShape: IRI, score: number }>. In that case there two things that change, the cardinality (1 vs N) and the type (IRI vs{widget: IRI, scoreShape: IRI, score: number}). I think it is helpful to have simple return types if possible.About 5. Yes I agree, however we did add a hint about using generators. I prefer the spec not having any JavaScript specific things, if possible. In my latest PR for the scoring system I do not need such things.
About 6. I agree, however we can also have multiple design where we can reasonably see that one is much better than the other. I think that is currently the case.
About 7. Yes indeed, the requirement is that this still works and I also prefer / want there only to be one system.
smessie commentedon Sep 28, 2026
Thanks @danielbeeke, that helps. A few follow-ups:
About 3. From your explanation, this is about a widget creator not having to write scoring shapes to hard-wire a widget. If so, I think that is already covered by requirement 2 ("create a custom widget and not touch any scoring graph triples"). The current wording of 3 says something different: that implementing a widget must not require RDF knowledge. I don't think that holds as a requirement for a SHACL UI specification, where everything a widget works with is RDF. Could you either merge 3 into 2, or rephrase it to say what you described above?
About 4. Agreed, avoiding union return types makes sense. The current wording ("simple singular return types") can easily be read as "a single value, not a list", which is not what you mean. How about: "Each scoring system function must have one fixed return type (no union types)."
About 5. Agreed. Note that the generator hint on
gh-pagesis already in a non-normative note, and I think that is the right place for implementation hints like this. We should drop the "MAY" from that note though, as RFC 2119 keywords do not belong in informative text.About 6. Could you clarify what "much better" means here: clarity, correctness, ease of implementation, performance? A specification weighs these differently than a production application or a reference implementation would. If we keep this requirement, I think it should name the criteria we use to compare designs. Otherwise it is hard to apply it to proposals.
About 7. Agreed, let's keep it as an explicit requirement so this behaviour is preserved, with the understanding that it is part of the scoring system and not a separate mechanism. Maybe something like: "If a widget declared with
shui:editororshui:vieweris not accepted, widget selection MUST continue with the regular scoring, without a separate fallback mechanism."danielbeeke commentedon Sep 28, 2026
About 3: In DASH it is possible for a web developer without prior knowledge of RDF to implement a custom widget. Actually using it does require adding a triple like
ex:shapeiri shui:editor ex:newWidget. I prefer to keep the required RDF knowledge to a minimum.About 4: Agree
About 5: I would like to see the hint removed. I rather have the algo be simple, such that the hint is not needed. My new implementation and PR are simpler and do not need this complicated stuff.
About 6 Ease of implementation as the functions defined do not called each other in ways that make their usage ambiguous. And performance, the score function is not called anymore in my implementation when we are just using the select function for the best widget.
I see these two competing ideas:
Idea 1: Reuse score within the select function:
Idea 2: Do not reuse the score function within the select function:
#1273
I argue that idea 2 is more performant because it does not execute unnecessary SHACL validations to get scores that are not used.
I also argue it is simpler and easier to implement. The two functions have clear goals:
About 7: good
Ill adjust the issue description accordingly.
smessie commentedon Sep 28, 2026
Thanks @danielbeeke.
About 3. I understand the intent: widget authors should need as little RDF knowledge as possible, and I agree with that goal. But a widget reads and writes RDF terms, so it has to handle IRIs versus literals, datatypes and language tags. Saying that implementing a widget requires no RDF knowledge doesn't hold for a SHACL UI specification. More importantly, the wording is still unclear to me, which makes it hard to discuss. What would this requirement concretely constrain in the spec? What spec text would violate it?
About 5. Fine with me. If the algorithm is clear without the note, it can go.
About 6. As a requirement, "reasonably optimal performance" still can't be tested. It has now become an argument for one design over the other, and I think that comparison belongs in #1249 and #1273 rather than in the requirements list. I'll try to have a look at the exact PRs as well. If we want a performance requirement, I'd make it concrete, for example: "The algorithm MUST allow the best widget to be determined without evaluating the matchers of lower-ranked widget scores."
smessie commentedon Oct 14, 2026
I think it is important to discuss the requirements in this issue here first, before delving into different implementations as PRs.
Can we settle on them here?
@danielbeeke I see you thumbs-upped my last comment, but did not update the OP accordingly. Do you want to discuss this further or do you already agree with my comments?
danielbeeke commentedon Oct 14, 2026
I updated the text, does this fit better for you?
danielbeeke commentedon Oct 14, 2026
@smessie lets agree on requirements as you proposed in the reeview of the PR's.
Your points:
I see a problem with these two:
only a score function in the spec
from my perspective they do two different things, maybe a little bit similar but not the same. Select gives the default selected widget, including the hard wiring. Score gives a list of scored widgets without hard coding.
I think we can do multiple things when shui:editor is set. It could mean do not show alternatives. It could also mean this is the default selected widget, and allow to change it.
I personally think it might be more logical to leave this to the implementation.
About preprocessing
I think its better to do the hard wiring with code. It is less code, score shape writers need to write less. It is just an IF statement with a reaction, while the preprocessing is more work in the implementation, you have to define an algorithm to make those score shapes correctly.
smessie commentedon Oct 14, 2026
Then the place that needs the default widget can take the first result of the score function. I don't see why the two should behave differently for hard-wired widgets. It feels like we're adding a separate function for each use case instead of finding the best design from the spec's point of view. Having two functions also creates inconsistencies. In #1273, a widget pinned with
shui:editorcan be the default widget (Select) and still not appear in the list of alternatives (Score). Hard-wiring should be supported in one function, and another widget score should still be able to override it.That's exactly why we need to be clear on the semantics of
shui:editor/shui:viewer. We've discussed this before, and that discussion produced the definition in the vocabulary:shui:editor"links a shape to the editor that is preferred for editing the values of that shape". It indicates a preference and doesn't mandate the widget. The shapes should still be valid when another widget score overrides it. Whether the user interface offers alternatives is a separate question, already covered by Widget Selection ("Implementations MAY allow users to switch to any widget included in the results"). If we also want a way to say "never show alternatives", that's a different feature, and it shouldn't be read intoshui:editor.I disagree, for reasons I've given before:
On the cost: preparation doesn't require score shape writers to write anything. That's requirement 2, a custom widget pinned without any scoring triples, and preparation generates the score for it. It also isn't "just an IF statement". Once the pinned widget also has to be accept-checked, fall through when rejected, appear in the list of alternatives and be overridable by a higher score, the IF has to replicate what scoring already does. That's the complexity we ran into in #994, and it's why we chose preparation there after our call. You also described it then as the right decision (#994 review).
That said, how an implementation achieves this is up to the implementation. An implementation that handles pinned widgets with an IF and gives them the default score is fine, as long as the result is the same as with preparation. What I want is for the spec to define the behaviour once, through the scoring mechanism, and not as a second path next to it.
smessie commentedon Oct 14, 2026
Thanks @danielbeeke. Requirements 4 and 7 look good to me now, and I agree with striking 3 and 6 (3 is now Value 1). A few remaining points:
5. We agreed that the spec should describe the algorithms in abstract terms and leave the programming language to implementations, but the wording hasn't been updated yet. I'd suggest: "The algorithms MUST be described in abstract terms, without concepts specific to a programming language." This matters now: #1273 uses
undefinedas a return value.Value 2. This brings back requirement 6 in another form. It judges the spec text as written, not implementations. So a spec that defines the full ordered list of widgets fails it by definition, even when every implementation stops at the first accepted widget. That decides the design question we're discussing in #1249 and #1273 in favour of one side. It's also listed under "not easily testable" but gives a hard threshold, and I don't know where the numbers come from (why 1000, why 20 fields, why only editors?). I'm not convinced we need a performance requirement at all, since I see this as up to implementations. But if we do want one, I'd use the concrete form I suggested earlier: "The algorithm MUST allow the best widget to be determined without evaluating the matchers of lower-ranked widget scores." That leaves the optimisation to implementations, and the current spec already allows it.
1. "Hard wire" conflicts with the vocabulary, which defines
shui:editor/shui:vieweras the preferred widget. As discussed above, I think we should settle this explicitly as a requirement: "It MUST be possible to declare a preferred widget on a shape withshui:editororshui:viewer. A widget score with a higher score MUST be able to override it."8. Removing all accept matchers doesn't give a closed world. It makes every widget with a score acceptable, which is more open, not less. What gives a closed world is removing the widget scores, so that only the declared widgets are left. I'd state the goal rather than a method: "It MUST be possible to restrict widget selection to the widgets declared in the shapes."
9. "The interface" can be read as the user interface or as the calling code. I think we agree the scoring system reports to the application, which then decides how to show it. #1273 turned this into a MUST on the user interface. Suggested wording: "If no widget is found, this is reported to the caller."
New. Following from the discussion above, I'd add: "The widget initially selected for a Property UI Component MUST be the first of the widgets the user can switch between." In #1273, a pinned widget can be initially selected but missing from the list of alternatives.
Minor. In 2, "scoring shape" should be "widget score" (
shui:WidgetScore), to match the spec's terminology.smessie commentedon Oct 15, 2026
Here is a proposal for how we could formulate the part of the scoring system we keep disagreeing on. It comes down to how
shui:editorandshui:viewertake part in widget selection. Apologies for the length of this comment. I wanted to explain the reasoning behind each decision, so that you can give focused feedback on the parts you disagree with.Describing the what, not the how
I can agree with @bergos and @danielbeeke that the spec should describe what the result of widget selection is, and leave how to compute it to implementations. The Scoring Graph Preparation section I added in #994 does the opposite. It prescribes a separate step that rewrites the scoring graph, and reads as a second component next to the scoring system. I'd replace it with a description of the result, roughly like this.
How an implementation achieves this is up to the implementation. It can check the preferred widget first with an IF statement, add the equivalent widget score to the scoring graph up front, or handle it inside the score function. All of these conform as long as the result is the same. This removes the Scoring Graph Preparation section and its algorithm from the spec.
Score conventions
This relies on the score bands in the Score Conventions section, which I'd keep. Score 40 means that the shape declares the widget. Every built-in widget already has a widget score of 40 that only applies when the shape prefers that specific widget, and no built-in widget scores higher. A third-party widget that follows the bands therefore never ties with or outranks the preferred widget by accident. A widget that should override the preferred widget uses a score above 40, deliberately.
Showing only the best widget needs at most one data validation
The concern was that an implementation that only shows the best widget would need many SHACL validations, even when every property shape declares a preferred widget. That isn't the case. All scores are known before any validation runs, because they are static values in the scoring graph. To find the best widget, an implementation only needs to consider the widget scores of 40 and higher, in descending order, and can stop at the first one that matches and is accepted.
With the built-in widgets, nothing scores above 40. The matcher function checks the shapes graph before the data graph, so the scores of 40 of all other widgets fail on whether the shape prefers them, without touching the data. That check depends only on the shapes graph, so it can be done once per shape and reused for every value. What remains is the score of 40 of the preferred widget itself. For 9 of the 26 built-in widgets it has no condition on the value, and for the other 17 it checks one, such as "the value is an IRI" for the IRI editor. The existing accept matchers only check the shapes graph. So the best widget costs at most one data validation per value, and none for an empty field.
That one validation is not overhead. It's what prevents a preferred IRI editor from being used for a literal. An IF statement has to answer the same question, or it selects a widget that can't handle the value. If the preferred widget doesn't fit the value, its score of 40 doesn't match and selection continues with the regular scoring, in line with requirement 7.
Preferred, not pinned
Requirement 7 already means a declared widget is not absolute. If its accept matcher rejects it, another widget is selected. A "pinned" widget that can still be overruled by an accept matcher is not really pinned. "Preferred" describes exactly this behaviour, and it's also how the vocabulary defines
shui:editor("the editor that is preferred for editing the values of that shape").The only remaining difference between pinned and preferred is whether a widget score with a higher score can override the declared widget. With the built-in widgets that never happens, because nothing scores above 40. So for everyone who doesn't add their own scores, pinned and preferred behave identically. The difference only shows when someone deliberately adds a higher score.
Why overriding matters
Daniel gave the use case himself in #1249. A community widget, say an AwesomeDatePicker, should become the default for dates by giving it a higher score. Now combine that with application profiles. Organisations adopt published profiles such as DCAT-AP-NL unmodified, and those shapes may declare
shui:editor shui:DatePickerEditorondcterms:issued. With preferred semantics, a deployment that installs the AwesomeDatePicker adds one widget score with score 50 to its own scoring graph, and the widget is used everywhere, including in the profile's shapes. With pinned semantics, the deployment has to modify or fork every shape that declares the built-in date picker, which is exactly what organisations avoid.The same applies to other deployment choices, such as replacing a widget with a more accessible one for a specific user group. The shapes graph is shared and often published, while the scoring graph is a separate input that belongs to the deployment (as agreed in #994). Pinning would let the shared shapes override the local deployment, while preferred semantics keep that decision local.
Open points