Validation report reporting on matched target nodes #1221

Activity

1. robert-david

   added

   CoreFor SHACL 1.2 Core specFor SHACL 1.2 Core spec

   on Aug 31, 2026
2. afs commented on Aug 31, 2026

   

   afs

   on Aug 31, 2026

   Contributor

   More actions

   I think having a way to record the total validation outcome could be useful when high confidence is required.

   Even if the activity records exactly which data and exactly which shapes were uses, reproducing the validation is not straight forward (software may not be accessible, many other factors).

   Reproducibility is currently a major issue in software delivery, down to getting the "same bytes" in archive files so they can be signed.
3. osavkovic commented on Aug 31, 2026

   

   osavkovic

   on Aug 31, 2026

   Last edited by osavkovic

   More actions

   Thanks Robert!

   I've raised the same question in several issues (#1127, #1199). I think it's crucial for combining validation with rules, and more generally for making SHACL validation usable as a step in a data-management pipeline.

   Another important bit is that I think we should say explicitly in the spec that you cannot obtain the conforming nodes by negating the violations (often people think otherwise). Small example:

   ex:PersonShape   sh:targetClass ex:Person .
   ex:CompanyShape  sh:targetClass ex:Company .
   
   ex:alice a ex:Person .    # conforms
   ex:bob   a ex:Person .    # violates
   ex:acme  a ex:Company .   # conforms

   The report contains one violation, for ex:bob. Take "all nodes minus violating nodes" and you get ex:alice, ex:acme, and every other node in the graph...

   Hence, I would argue for a standardized report. E.g., something of the form

   < node, shape, true/false >
   

   The way I like to think about it: validation takes as input an assertion of what should hold — the targets, i.e. < node, shape, true > — and should output what actually holds when checked, i.e. < node, shape, true/false >. As of now, we are getting only half, and that for the targeted nodes only.

   Regarding the stats, I guess one can easily obtain them with a single SPARQL query once we have full data on conformance/violation. Also note that a node can be a focus node for several shapes, and conform to one while violating another.
4. ajnelson-nist commented on Aug 31, 2026

   

   ajnelson-nist

   on Aug 31, 2026

   Contributor

   More actions

   [] sh:conforms: true . has been a pretty satisfactory result to date. [] sh:conforms true ; sh:matchedNodeCount 12345 . would be a significant comfort while reminding the reviewer of the possibility of [] sh:conforms true ; sh:matchedNodeCount 0 ..

   A report of all node-shape pairs feels like an "opt-in" feature. Most of my day-to-day SHACL review would suffer significantly if I'm overwhelmed by "...and Alice matched and passed ex:PersonShape...", because I'm looking for the rare typo or interesting inference issue.

   The "standardized report" combining a node, shape, and pass-status sounds like it would be a sibling node to a sh:ValidationResult in a sh:ValidationReprt. The false cases would be redundant with sh:ValidationResult, because the focus node is reported there. And the true cases would be helpful if one really needed to confirm the node matched the shape. But, again, I'd really rather that be opt-in.
5. afs commented on Aug 31, 2026

   

   afs

   on Aug 31, 2026

   Contributor

   More actions

   A report of all node-shape pairs feels like an "opt-in" feature.

   I agree it should not swap the validation report i.e. not one positive with the same subject as rdf:type sh:ValidationReport.
   It could be a link to the "standardized report" (subgraph) or separate document.
6. robert-david commented on Aug 31, 2026

   

   robert-david

   on Aug 31, 2026

   ContributorAuthor

   More actions

   I agree we should make positive reports as optional addition only to not break backwards compatibility.
   @ajnelson-nist would sh:matchedNodeCount be the total focus node count or the 'positive', i.e. conformant, matches?
   Should we have 3 count properties for number of target nodes, number of conformant nodes and number of violating nodes?

   @afs @osavkovic to have an optional report such as < node, shape, true/false >, we can either have i) one property to link to a subgraph or have ii) a similar property as sh:result linking all positive matches (individually). Both would be non-breaking changes.

   What would be the common use cases? I see the main use in avoiding processing errors by making sure the shapes actually hit the target nodes, which would only require a count and not a detailed report on positive matches.
7. ajnelson-nist commented on Aug 31, 2026

   

   ajnelson-nist

   on Aug 31, 2026

   Contributor

   More actions

   [...] @ajnelson-nist would sh:matchedNodeCount be the total focus node count or the 'positive', i.e. conformant, matches? [...]

   I was imagining total focus node count.

   [...] Should we have 3 count properties for number of target nodes, number of conformant nodes and number of violating nodes? [...]

   Counting could be tricky. Say I have one node violate two shapes. Would it only be counted once in the violating-nodes tally?

   [...] What would be the common use cases? I see the main use in avoiding processing errors by making sure the shapes actually hit the target nodes, which would only require a count and not a detailed report on positive matches.

   I think I would appreciate the use case of confirming my shapes reviewed something rather than claim trivial conformance of reviewing nothing.

   I don't think I would personally get value from knowing "positive" matches in addition to total matches. What would I do with the information? sh:ValidationResults give me something to react to.
8. osavkovic commented on Aug 31, 2026

   

   osavkovic

   on Aug 31, 2026

   More actions

   @robert-david

   Option ii) looks closer to the report style, I guess.

   Regarding the possible use cases, something I can imagine :

   • pipelining with the rules. We can now read the outcome of the shape validation and "query" it directly in the rules. That would be useful in applications where rules are used together for data repair or standardization, or even mapping and transformation between datasets (I recall there you had some issues on that, cannot find them now).
   • data completion. For the checked data we often want to add some mark or annotation, e.g. that a node was validated against a given shape and conformed. Today we can only annotate what failed.
   • shapes debugging. I feel that most people do not write shapes that reference other shapes, or it's mostly depth one. (I believe it's because SHACL is lacking proper tutorials on data modeling, or even further some sort of normal forms like in SQL.) Anyhow, once we have more complex shapes one would like to analyze where the error comes from, and what has been valid so far and what not. You can say complex shape dependencies rarely occur in practice, but I would say that's because people are not explained how to model their data and write more complex SHACL freely.
9. HolgerKnublauch commented on Sep 1, 2026

   

   HolgerKnublauch

   on Sep 1, 2026

   Contributor

   More actions

   FWIW here is an optional report that people can produce in our product when running the validation

   

   (continued)

   

   This report is possible because we have tweaked the engine to collect all that info. Of course it would be ideal if a platform-neutral vocabulary can be agreed on for all that info.

   The dash namespace also contains a class

   dash:SuccessResult
     a rdfs:Class ;
     rdfs:comment "A result representing a successfully validated constraint." ;
     rdfs:label "Success result" ;
     rdfs:subClassOf sh:AbstractResult ;
   .
   

   which then points to the focus node that passed all constraints.

   I welcome a PR that would make the discussion here a bit more concrete.

   👍React with 👍1Reacted by Robert David
10. ajnelson-nist commented on Sep 17, 2026

    

    ajnelson-nist

    on Sep 17, 2026

    Contributor

    More actions

    @labra - I hear from @nicholascar that you may be interested in this issue.