Merge pull request #2047 from Plastic-time/bugfix/r3.12-graphql-type-system-definitions

A GraphQL document that also declared a type-system definition without braces (`scalar`, `union`, `directive`, `extend`) or began with a description lost the operations after it, because the reader looked for a brace that never came. Those definitions and descriptions are now stepped over by their own grammar.
This commit is contained in:
BFlat authored and GitHub committed 2026-11-18 05:55:36 +07:00
commit bc6198ee75
4 files changed
+397 -81

No files matched your search

+297 -81
View File
@@ -32,6 +32,10 @@ pub struct Document {
pub open_ended: bool,
}
/// Keywords that may open a top-level GraphQL executable or type-system definition.
const DOC_KEYWORDS: &[&str] =
&["query", "mutation", "subscription", "fragment", "type", "schema", "extend", "input", "interface", "union", "enum", "scalar", "directive"];
fn is_name_start(c: char) -> bool {
c.is_ascii_alphabetic() || c == '_'
}
@@ -40,6 +44,248 @@ fn is_name_char(c: char) -> bool {
c.is_ascii_alphanumeric() || c == '_'
}
/// Skip GraphQL ignored tokens (`Whitespace`, `,`, BOM, `#` comments) and optionally [`HOLE`]s.
/// Returns `true` when at least one [`HOLE`] was skipped.
fn skip_trivia(chars: &mut std::iter::Peekable<std::str::CharIndices<'_>>, allow_hole: bool) -> bool {
let mut saw_hole = false;
while let Some(&(_, c)) = chars.peek() {
if allow_hole && c == HOLE {
saw_hole = true;
chars.next();
} else if c == '#' {
for (_, d) in chars.by_ref() {
if d == '\n' || d == '\r' {
break;
}
}
} else if c.is_whitespace() || c == ',' || c == '\u{feff}' {
chars.next();
} else {
break;
}
}
saw_hole
}
/// Read a GraphQL `Name` token starting at the current cursor position.
fn read_name<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<&'a str> {
let &(start, first) = chars.peek()?;
if !is_name_start(first) {
return None;
}
let mut end = start;
while let Some(&(i, c)) = chars.peek() {
if !is_name_char(c) {
break;
}
end = i + c.len_utf8();
chars.next();
}
text.get(start..end)
}
/// Step over a balanced `(...)` argument list, skipping strings and comments inside.
fn skip_parens(chars: &mut std::iter::Peekable<std::str::CharIndices<'_>>, text: &str) -> Option<()> {
chars.peek().filter(|&&(_, c)| c == '(')?;
let mut depth = 0usize;
while let Some(&(_, c)) = chars.peek() {
match c {
'(' => {
depth = depth.checked_add(1)?;
chars.next();
}
')' => {
depth = depth.checked_sub(1)?;
chars.next();
if depth == 0 {
return Some(());
}
}
'"' => skip_string(chars, text)?,
'#' => {
for (_, d) in chars.by_ref() {
if d == '\n' || d == '\r' {
break;
}
}
}
_ => {
chars.next();
}
}
}
None
}
/// Step over zero or more `@directive(...)` annotations, returning how many were read.
fn skip_directives<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<usize> {
let mut count = 0usize;
loop {
let _ = skip_trivia(chars, false);
let Some(&(_, '@')) = chars.peek() else { return Some(count) };
chars.next();
read_name(chars, text)?;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '(') {
skip_parens(chars, text)?;
}
count += 1;
}
}
/// Step over a `{ ... }` selection or field block; requires closing `}` and at least one named member/hole.
fn skip_braced_block<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<()> {
chars.peek().filter(|&&(_, c)| c == '{')?;
let mut depth = 0usize;
let mut selected = false;
while let Some(&(_, c)) = chars.peek() {
match c {
'{' => depth = depth.checked_add(1)?,
'}' => {
depth = depth.checked_sub(1)?;
if depth == 0 {
chars.next();
return selected.then_some(());
}
}
'"' => {
skip_string(chars, text)?;
continue;
}
'#' => {
for (_, d) in chars.by_ref() {
if d == '\n' || d == '\r' {
break;
}
}
continue;
}
c if is_name_start(c) || c == HOLE => selected = true,
_ => {}
}
chars.next();
}
None
}
/// Step over a `scalar Name Directives?` tail after the `scalar` keyword.
fn skip_scalar_def<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<()> {
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
skip_directives(chars, text).map(|_| ())
}
/// Step over a `union Name Directives? (= |? Type (| Type)*)?` tail after the `union` keyword.
fn skip_union_def<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<()> {
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
skip_directives(chars, text)?;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '=') {
chars.next();
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '|') {
chars.next();
let _ = skip_trivia(chars, false);
}
read_name(chars, text)?;
loop {
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '|') {
chars.next();
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
} else {
break;
}
}
}
Some(())
}
/// Step over a `directive @Name (Args)? repeatable? on |? Loc (| Loc)*` tail after `directive`.
fn skip_directive_def<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<()> {
let _ = skip_trivia(chars, false);
chars.peek().filter(|&&(_, c)| c == '@')?;
chars.next();
read_name(chars, text)?;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '(') {
skip_parens(chars, text)?;
let _ = skip_trivia(chars, false);
}
let mut word = read_name(chars, text)?;
if word == "repeatable" {
let _ = skip_trivia(chars, false);
word = read_name(chars, text)?;
}
if word != "on" {
return None;
}
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '|') {
chars.next();
let _ = skip_trivia(chars, false);
}
read_name(chars, text)?;
loop {
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '|') {
chars.next();
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
} else {
break;
}
}
Some(())
}
/// Step over an `extend ...` type-system extension after the `extend` keyword.
fn skip_extend_def<'a>(chars: &mut std::iter::Peekable<std::str::CharIndices<'a>>, text: &'a str) -> Option<()> {
let _ = skip_trivia(chars, false);
match read_name(chars, text)? {
"scalar" => skip_scalar_def(chars, text),
"union" => skip_union_def(chars, text),
"schema" => {
let dirs = skip_directives(chars, text)?;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '{') { skip_braced_block(chars, text) } else { (dirs > 0).then_some(()) }
}
"type" | "interface" | "input" | "enum" => {
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
let _ = skip_trivia(chars, false);
let mut has_clause = false;
let mut lookahead = chars.clone();
if read_name(&mut lookahead, text) == Some("implements") {
*chars = lookahead;
has_clause = true;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '&') {
chars.next();
let _ = skip_trivia(chars, false);
}
read_name(chars, text)?;
loop {
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '&') {
chars.next();
let _ = skip_trivia(chars, false);
read_name(chars, text)?;
} else {
break;
}
}
}
let dirs = skip_directives(chars, text)?;
let _ = skip_trivia(chars, false);
if chars.peek().is_some_and(|&(_, c)| c == '{') { skip_braced_block(chars, text) } else { (has_clause || dirs > 0).then_some(()) }
}
_ => None,
}
}
/// Read the definitions of a GraphQL document. `None` when the text isn't one: it must start with a
/// definition, and its brackets and strings must close. Fragments count as definitions but aren't
/// operations. `{ a }` is valid GraphQL, but also plenty of other things, so it counts only when
@@ -52,49 +298,51 @@ pub fn read_document(text: &str, allow_shorthand: bool) -> Option<Document> {
let mut chars = text.char_indices().peekable();
let mut definitions = 0usize;
loop {
// Between definitions.
let mut hole_here = false;
while let Some(&(_, c)) = chars.peek() {
if c == HOLE {
hole_here = true;
chars.next();
} else if c == '#' {
for (_, c) in chars.by_ref() {
if c == '\n' || c == '\r' {
break;
}
}
} else if c.is_whitespace() || c == ',' || c == '\u{feff}' {
chars.next();
} else {
break;
}
}
if hole_here {
if skip_trivia(&mut chars, true) {
doc.open_ended = true;
}
let Some(&(start, first)) = chars.peek() else { break };
let Some(&(_, first)) = chars.peek() else { break };
let had_description = if first == '"' {
skip_string(&mut chars, text)?;
let _ = skip_trivia(&mut chars, false);
true
} else {
false
};
let &(_, first) = chars.peek()?;
let (kind, is_operation) = if first == '{' {
if !allow_shorthand {
if !allow_shorthand || had_description {
return None;
}
("query", true)
} else if is_name_start(first) {
let mut end = start;
while let Some(&(i, c)) = chars.peek() {
if !is_name_char(c) {
break;
}
end = i + c.len_utf8();
chars.next();
}
match text.get(start..end)? {
match read_name(&mut chars, text)? {
"query" => ("query", true),
"mutation" => ("mutation", true),
"subscription" => ("subscription", true),
"fragment" => ("fragment", false),
// Type definitions and the like are GraphQL too, but say nothing a client calls.
"type" | "schema" | "extend" | "input" | "interface" | "union" | "enum" | "scalar" | "directive" => ("other", false),
"scalar" => {
skip_scalar_def(&mut chars, text)?;
definitions += 1;
continue;
}
"union" => {
skip_union_def(&mut chars, text)?;
definitions += 1;
continue;
}
"directive" => {
skip_directive_def(&mut chars, text)?;
definitions += 1;
continue;
}
"extend" if !had_description => {
skip_extend_def(&mut chars, text)?;
definitions += 1;
continue;
}
// Braced type-system definitions are GraphQL too, though they name no client operation.
"type" | "schema" | "input" | "interface" | "enum" => ("other", false),
_ => return None,
}
} else {
@@ -106,7 +354,7 @@ pub fn read_document(text: &str, allow_shorthand: bool) -> Option<Document> {
let mut name_is_hole = false;
let mut saw_anything = false;
let mut parens = 0usize;
while let Some(&(i, c)) = chars.peek() {
while let Some(&(_, c)) = chars.peek() {
// Ignored tokens may separate any header tokens, including the name.
if c == '#' {
for (_, d) in chars.by_ref() {
@@ -135,15 +383,7 @@ pub fn read_document(text: &str, allow_shorthand: bool) -> Option<Document> {
_ => {}
}
if !saw_anything && is_name_start(c) && is_operation {
let mut end = i;
while let Some(&(j, d)) = chars.peek() {
if !is_name_char(d) {
break;
}
end = j + d.len_utf8();
chars.next();
}
name = text.get(i..end).map(String::from);
name = read_name(&mut chars, text).map(String::from);
saw_anything = true;
// A name running straight into run-time text isn't all of the name.
if chars.peek().is_some_and(|&(_, d)| d == HOLE) {
@@ -157,42 +397,7 @@ pub fn read_document(text: &str, allow_shorthand: bool) -> Option<Document> {
}
chars.next();
}
// The selection set: it must close, and must select something.
chars.peek().filter(|&&(_, c)| c == '{')?;
let mut depth = 0usize;
let mut selected = false;
let mut closed = false;
while let Some(&(_, c)) = chars.peek() {
match c {
'{' => depth = depth.checked_add(1)?,
'}' => {
depth = depth.checked_sub(1)?;
if depth == 0 {
chars.next();
closed = true;
break;
}
}
'"' => {
skip_string(&mut chars, text)?;
continue;
}
'#' => {
for (_, d) in chars.by_ref() {
if d == '\n' || d == '\r' {
break;
}
}
continue;
}
c if is_name_start(c) || c == HOLE => selected = true,
_ => {}
}
chars.next();
}
if !closed || !selected {
return None;
}
skip_braced_block(&mut chars, text)?;
definitions += 1;
if is_operation && doc.operations.len() < MAX_DOC_OPERATIONS {
doc.operations.push(DocOperation { kind, literal: !name_is_hole, name: if name_is_hole { None } else { name } });
@@ -246,10 +451,21 @@ fn document_start(text: &str) -> &str {
}
/// Could this text be a whole document? A quick test before the real read, on every string.
/// A document may begin with a comment, as a generated client often writes one.
/// A document may begin with a comment or GraphQL description string.
pub fn may_be_document(text: &str) -> bool {
let t = document_start(text);
(t.starts_with("query") || t.starts_with("mutation") || t.starts_with("subscription") || t.starts_with("fragment")) && t.contains('{')
if !text.contains('{') {
return false;
}
let mut t = document_start(text);
if t.starts_with('"') {
let mut chars = t.char_indices().peekable();
if skip_string(&mut chars, t).is_none() {
return false;
}
let after = chars.peek().map_or(t.len(), |&(i, _)| i);
t = document_start(t.get(after..).unwrap_or(""));
}
DOC_KEYWORDS.iter().any(|kw| t.strip_prefix(kw).is_some_and(|rest| !rest.chars().next().is_some_and(is_name_char)))
}
/// What a URL says about an API, as `(protocol, procedures)`: a GraphQL endpoint, or a tRPC one with
+97
View File
@@ -0,0 +1,97 @@
//! Regression tests for GraphQL documents mixing SDL definitions or descriptions with operations.
use craft_anything_js::api_ops::{may_be_document, read_document};
use craft_anything_js::{Limits, analyze};
/// Documents with braced `type` or `fragment` definitions before operations continue to extract operations.
#[test]
fn braced_type_and_fragment_definitions_preceding_query_work() {
let doc = read_document("type User { id: ID! }\nfragment Fields on User { id }\nquery GetUser { user { ...Fields } }", false).unwrap();
assert_eq!(doc.operations.len(), 1);
assert_eq!(doc.operations[0].kind, "query");
assert_eq!(doc.operations[0].name.as_deref(), Some("GetUser"));
}
/// Braceless SDL definitions (`scalar`, `union`, `directive`, `extend`) must not swallow following operations.
#[test]
fn braceless_sdl_definitions_do_not_swallow_operations() {
let cases = [
("scalar DateTime\nquery GetFeed { feed { createdAt } }", "query", Some("GetFeed")),
(
"scalar JSON @specifiedBy(url: \"https:// scalars.graphql.org/andimarek/json\")\nmutation Save($p: JSON!) { save(payload: $p) }",
"mutation",
Some("Save"),
),
("union SearchResult = User | Post\nquery Search($q: String!) { search(q: $q) { __typename } }", "query", Some("Search")),
(
"directive @client(always: Boolean = false) repeatable on FIELD | FRAGMENT_SPREAD\nquery LocalState { me @client { id } }",
"query",
Some("LocalState"),
),
(
"extend scalar DateTime @tag(name: \"iso\")\nextend union SearchResult = Comment\nextend schema @link(url: \"https://specs.apollo.dev/federation/v2.0\")\nquery Fed { me { id } }",
"query",
Some("Fed"),
),
];
for (src, expected_kind, expected_name) in cases {
assert!(may_be_document(src), "may_be_document should accept mixed SDL + operation:\n{src}");
let doc = read_document(src, false).unwrap_or_else(|| panic!("expected valid GraphQL document:\n{src}"));
assert_eq!(doc.operations.len(), 1, "expected 1 operation in:\n{src}\ngot: {doc:?}");
assert_eq!(doc.operations[0].kind, expected_kind);
assert_eq!(doc.operations[0].name.as_deref(), expected_name);
}
// Trailing braceless SDL after an operation must also remain a valid document.
let trailing = "query GetUser { user { id } }\nscalar DateTime\ndirective @client on FIELD";
let doc = read_document(trailing, false).expect("trailing SDL definitions are valid in a GraphQL document");
assert_eq!(doc.operations.len(), 1);
assert_eq!(doc.operations[0].name.as_deref(), Some("GetUser"));
}
/// Leading GraphQL `StringValue` descriptions (`"..."` and `"""..."""`) on definitions are accepted.
#[test]
fn leading_graphql_descriptions_are_accepted_without_loosening_prose_rejection() {
let described = "\"\"\"\nFetches the signed-in account.\n\"\"\"\nquery CurrentAccount { viewer { id } }";
assert!(may_be_document(described));
let doc = read_document(described, false).expect("block-string description before query is valid GraphQL");
assert_eq!(doc.operations.len(), 1);
assert_eq!(doc.operations[0].name.as_deref(), Some("CurrentAccount"));
let described_sdl = "\"ISO-8601 timestamp\"\nscalar DateTime\n\"Account lookup\"\nmutation UpdateTime($t: DateTime!) { setTime(t: $t) }";
assert!(may_be_document(described_sdl));
let doc2 = read_document(described_sdl, false).expect("single-line descriptions in mixed document are valid");
assert_eq!(doc2.operations.len(), 1);
assert_eq!(doc2.operations[0].name.as_deref(), Some("UpdateTime"));
// Plain English prose or non-GraphQL strings starting with SDL words or quotes must still be rejected.
for non_graphql in [
"scalar the mountain { fast }",
"union the workers { now }",
"directive to all hands { read }",
"extend the warranty { days }",
"\"just a quoted string with { braces }\"",
"scalar",
"union A =",
"directive @client",
] {
assert!(read_document(non_graphql, false).is_none(), "must reject non-GraphQL text: {non_graphql}");
}
}
/// End-to-end JS analysis recovers GraphQL operations from both string literals and `gql` templates with SDL headers.
#[test]
fn js_analysis_extracts_operations_from_sdl_prefixed_documents() {
let js = r#"
const A = gql`
directive @client on FIELD
query ReadCart { cart @client { total } }
`;
const B = "scalar Upload\nmutation UploadAvatar($f: Upload!) { uploadAvatar(file: $f) { ok } }";
"#;
let out = analyze("app.js", js, &Limits::default());
let ops: Vec<(&str, Option<&str>)> = out.facts.api_operations.iter().map(|o| (o.kind.as_str(), o.name.as_deref())).collect();
assert!(ops.contains(&("query", Some("ReadCart"))), "{ops:?}");
assert!(ops.contains(&("mutation", Some("UploadAvatar"))), "{ops:?}");
}
+2
View File
@@ -2190,6 +2190,8 @@ Owner: open · Status: in progress
— claimed: Plastic-time, branch bugfix/r3.12-graphql-string-line-breaks
- [x] Keep escaped triple quotes inside GraphQL block strings when reading operations.
— claimed: Plastic-time, branch bugfix/r3.12-graphql-block-strings
- [x] Keep GraphQL operations in documents that also declare braceless type-system definitions (`scalar`, `union`, `directive`, `extend`) or leading descriptions.
— claimed: Plastic-time, branch bugfix/r3.12-graphql-type-system-definitions
- [x] Read `+` in JavaScript constant text as JavaScript does: numbers add (`'/v' + (1 + 2)` is `/v3`, not `/v12`) and number literals read as their value (`0x10` is `16`).
— claimed: Plastic-time, branch bugfix/r3.12-js-numeric-addition
- [x] Walk past the old GNU sparse extension blocks to list the next TAR member.
+1
View File
@@ -324,6 +324,7 @@ coverage and secrets passed. The companion CLI was built with locked dependencie
| Skip comments in GraphQL operation headers | Skip GraphQL header comments before reading operation names and selection sets. | JavaScript crate regression tests on Windows cover corrected behavior and retained controls. No browser or live application run. |
| GraphQL block strings | Escaped triple quotes stay inside block strings, so operation names survive in queries with block string arguments and variable defaults. | Regressions with argument text, variable defaults and an unfinished string. |
| GraphQL quoted string boundaries | A regular quoted string cannot supply operation evidence if it contains a physical line break. Block strings and textual escapes still read. | Fixtures cover carriage returns, line feeds, backslash continuations and valid strings. |
| GraphQL SDL definitions and descriptions | Mixed GraphQL documents step past braceless `scalar`, `union`, `directive` and `extend` definitions and leading string descriptions without swallowing adjacent operations, while still rejecting ordinary prose and incomplete headers. | Four regressions in `graphql_sdl_operations` cover mixed SDL documents, trailing SDL definitions, single-line and block descriptions, prose rejection and end-to-end JavaScript analysis; three fail before the fix. |
| API endpoints in URL paths | GraphQL and tRPC endpoints are read from a URL's path only. The host ends at the first `/`, `?` or `#`, so a URL with no path whose query or fragment holds a slash (`https://auth.example.test?next=/graphql`, `#next=/trpc/user.list`) is no longer a GraphQL endpoint or tRPC procedure. A scheme counts only before any `/`, `?` or `#`, so `/api/graphql?next=https://x.test` stays a relative path. Run-time text in a partly literal URL (`https://{document.querySelector(#api).value}/graphql`) ends neither the host nor the path, and a `://` or `/` inside it is neither a scheme nor a segment boundary. | Regression tests: six such URLs, including a redirect to a real GraphQL URL in the query, give no API; paths with a query or fragment after them (`/graphql?next=/home`, `/v1/gql#top`, `/api/trpc/user.list?batch=1`) still do, as do relative paths, also with `https://` in their query, where `main` found none; `?`, `#` and `/` inside `{…}` in the host or path end nothing, and `{proto}://{host}/graphql` still reads; a hole holding `https://…?x=1` before the real `://`, or `/trpc/user.list` inside a path, no longer moves the scheme or adds a tRPC procedure; end to end, `fetch` of the two redirect URLs records no API operations and a real endpoint still does. The serialized analysis of 5,593 local JavaScript files was identical before and after. |
| Webpack recovery with direct eval | Factory parameter names are preserved when a possible direct eval call can inspect them, including inside nested functions. Indirect eval still permits renaming. | Four regression groups cover direct and dynamic eval arguments, parentheses, escaped identifier spelling, nested functions, optional/indirect calls, property calls and text-only mentions. The baseline failed three groups. A recovered synthetic factory threw ReferenceError under Node.js before the guard and returns 42 after it. |
| JavaScript fact line numbers | CR, LF, CRLF, U+2028 and U+2029 advance source locations correctly. | Regression tests cover parsed/fallback imports, mixed terminators, multibyte text and existing LF/CRLF controls. |