jcs.org
qemu with hax to log dma reads & writes
jcs.org/2018/11/12/vfio
1= How to use the QAPI code generator =
2
3Copyright IBM Corp. 2011
4Copyright (C) 2012-2016 Red Hat, Inc.
5
6This work is licensed under the terms of the GNU GPL, version 2 or
7later. See the COPYING file in the top-level directory.
8
9== Introduction ==
10
11QAPI is a native C API within QEMU which provides management-level
12functionality to internal and external users. For external
13users/processes, this interface is made available by a JSON-based wire
14format for the QEMU Monitor Protocol (QMP) for controlling qemu, as
15well as the QEMU Guest Agent (QGA) for communicating with the guest.
16The remainder of this document uses "Client JSON Protocol" when
17referring to the wire contents of a QMP or QGA connection.
18
19To map Client JSON Protocol interfaces to the native C QAPI
20implementations, a JSON-based schema is used to define types and
21function signatures, and a set of scripts is used to generate types,
22signatures, and marshaling/dispatch code. This document will describe
23how the schemas, scripts, and resulting code are used.
24
25
26== QMP/Guest agent schema ==
27
28A QAPI schema file is designed to be loosely based on JSON
29(http://www.ietf.org/rfc/rfc7159.txt) with changes for quoting style
30and the use of comments; a QAPI schema file is then parsed by a python
31code generation program. A valid QAPI schema consists of a series of
32top-level expressions, with no commas between them. Where
33dictionaries (JSON objects) are used, they are parsed as python
34OrderedDicts so that ordering is preserved (for predictable layout of
35generated C structs and parameter lists). Ordering doesn't matter
36between top-level expressions or the keys within an expression, but
37does matter within dictionary values for 'data' and 'returns' members
38of a single expression. QAPI schema input is written using 'single
39quotes' instead of JSON's "double quotes" (in contrast, Client JSON
40Protocol uses no comments, and while input accepts 'single quotes' as
41an extension, output is strict JSON using only "double quotes"). As
42in JSON, trailing commas are not permitted in arrays or dictionaries.
43Input must be ASCII (although QMP supports full Unicode strings, the
44QAPI parser does not). At present, there is no place where a QAPI
45schema requires the use of JSON numbers or null.
46
47
48=== Comments ===
49
50Comments are allowed; anything between an unquoted # and the following
51newline is ignored.
52
53A multi-line comment that starts and ends with a '##' line is a
54documentation comment. These are parsed by the documentation
55generator, which recognizes certain markup detailed below.
56
57
58==== Documentation markup ====
59
60Comment text starting with '=' is a section title:
61
62 # = Section title
63
64Double the '=' for a subsection title:
65
66 # == Subsection title
67
68'|' denotes examples:
69
70 # | Text of the example, may span
71 # | multiple lines
72
73'*' starts an itemized list:
74
75 # * First item, may span
76 # multiple lines
77 # * Second item
78
79You can also use '-' instead of '*'.
80
81A decimal number followed by '.' starts a numbered list:
82
83 # 1. First item, may span
84 # multiple lines
85 # 2. Second item
86
87The actual number doesn't matter. You could even use '*' instead of
88'2.' for the second item.
89
90Lists can't be nested. Blank lines are currently not supported within
91lists.
92
93Additional whitespace between the initial '#' and the comment text is
94permitted.
95
96*foo* and _foo_ are for strong and emphasis styles respectively (they
97do not work over multiple lines). @foo is used to reference a name in
98the schema.
99
100Example:
101
102##
103# = Section
104# == Subsection
105#
106# Some text foo with *strong* and _emphasis_
107# 1. with a list
108# 2. like that
109#
110# And some code:
111# | $ echo foo
112# | -> do this
113# | <- get that
114#
115##
116
117
118==== Expression documentation ====
119
120Each expression that isn't an include directive may be preceded by a
121documentation block. Such blocks are called expression documentation
122blocks.
123
124When documentation is required (see pragma 'doc-required'), expression
125documentation blocks are mandatory.
126
127The documentation block consists of a first line naming the
128expression, an optional overview, a description of each argument (for
129commands and events) or member (for structs, unions and alternates),
130and optional tagged sections.
131
132FIXME: the parser accepts these things in almost any order.
133
134Extensions added after the expression was first released carry a
135'(since x.y.z)' comment.
136
137A tagged section starts with one of the following words:
138"Note:"/"Notes:", "Since:", "Example"/"Examples", "Returns:", "TODO:".
139The section ends with the start of a new section.
140
141A 'Since: x.y.z' tagged section lists the release that introduced the
142expression.
143
144For example:
145
146##
147# @BlockStats:
148#
149# Statistics of a virtual block device or a block backing device.
150#
151# @device: If the stats are for a virtual block device, the name
152# corresponding to the virtual block device.
153#
154# @node-name: The node name of the device. (since 2.3)
155#
156# ... more members ...
157#
158# Since: 0.14.0
159##
160{ 'struct': 'BlockStats',
161 'data': {'*device': 'str', '*node-name': 'str',
162 ... more members ... } }
163
164##
165# @query-blockstats:
166#
167# Query the @BlockStats for all virtual block devices.
168#
169# @query-nodes: If true, the command will query all the
170# block nodes ... explain, explain ... (since 2.3)
171#
172# Returns: A list of @BlockStats for each virtual block devices.
173#
174# Since: 0.14.0
175#
176# Example:
177#
178# -> { "execute": "query-blockstats" }
179# <- {
180# ... lots of output ...
181# }
182#
183##
184{ 'command': 'query-blockstats',
185 'data': { '*query-nodes': 'bool' },
186 'returns': ['BlockStats'] }
187
188==== Free-form documentation ====
189
190A documentation block that isn't an expression documentation block is
191a free-form documentation block. These may be used to provide
192additional text and structuring content.
193
194
195=== Schema overview ===
196
197The schema sets up a series of types, as well as commands and events
198that will use those types. Forward references are allowed: the parser
199scans in two passes, where the first pass learns all type names, and
200the second validates the schema and generates the code. This allows
201the definition of complex structs that can have mutually recursive
202types, and allows for indefinite nesting of Client JSON Protocol that
203satisfies the schema. A type name should not be defined more than
204once. It is permissible for the schema to contain additional types
205not used by any commands or events in the Client JSON Protocol, for
206the side effect of generated C code used internally.
207
208There are eight top-level expressions recognized by the parser:
209'include', 'pragma', 'command', 'struct', 'enum', 'union',
210'alternate', and 'event'. There are several groups of types: simple
211types (a number of built-in types, such as 'int' and 'str'; as well as
212enumerations), complex types (structs and two flavors of unions), and
213alternate types (a choice between other types). The 'command' and
214'event' expressions can refer to existing types by name, or list an
215anonymous type as a dictionary. Listing a type name inside an array
216refers to a single-dimension array of that type; multi-dimension
217arrays are not directly supported (although an array of a complex
218struct that contains an array member is possible).
219
220All names must begin with a letter, and contain only ASCII letters,
221digits, hyphen, and underscore. There are two exceptions: enum values
222may start with a digit, and names that are downstream extensions (see
223section Downstream extensions) start with underscore.
224
225Names beginning with 'q_' are reserved for the generator, which uses
226them for munging QMP names that resemble C keywords or other
227problematic strings. For example, a member named "default" in qapi
228becomes "q_default" in the generated C code.
229
230Types, commands, and events share a common namespace. Therefore,
231generally speaking, type definitions should always use CamelCase for
232user-defined type names, while built-in types are lowercase.
233
234Type names ending with 'Kind' or 'List' are reserved for the
235generator, which uses them for implicit union enums and array types,
236respectively.
237
238Command names, and member names within a type, should be all lower
239case with words separated by a hyphen. However, some existing older
240commands and complex types use underscore; when extending such
241expressions, consistency is preferred over blindly avoiding
242underscore.
243
244Event names should be ALL_CAPS with words separated by underscore.
245
246Member names starting with 'has-' or 'has_' are reserved for the
247generator, which uses them for tracking optional members.
248
249Any name (command, event, type, member, or enum value) beginning with
250"x-" is marked experimental, and may be withdrawn or changed
251incompatibly in a future release.
252
253Pragma 'name-case-whitelist' lets you violate the rules on use of
254upper and lower case. Use for new code is strongly discouraged.
255
256In the rest of this document, usage lines are given for each
257expression type, with literal strings written in lower case and
258placeholders written in capitals. If a literal string includes a
259prefix of '*', that key/value pair can be omitted from the expression.
260For example, a usage statement that includes '*base':STRUCT-NAME
261means that an expression has an optional key 'base', which if present
262must have a value that forms a struct name.
263
264
265=== Built-in Types ===
266
267The following types are predefined, and map to C as follows:
268
269 Schema C JSON
270 str char * any JSON string, UTF-8
271 number double any JSON number
272 int int64_t a JSON number without fractional part
273 that fits into the C integer type
274 int8 int8_t likewise
275 int16 int16_t likewise
276 int32 int32_t likewise
277 int64 int64_t likewise
278 uint8 uint8_t likewise
279 uint16 uint16_t likewise
280 uint32 uint32_t likewise
281 uint64 uint64_t likewise
282 size uint64_t like uint64_t, except StringInputVisitor
283 accepts size suffixes
284 bool bool JSON true or false
285 null QNull * JSON null
286 any QObject * any JSON value
287 QType QType JSON string matching enum QType values
288
289
290=== Include directives ===
291
292Usage: { 'include': STRING }
293
294The QAPI schema definitions can be modularized using the 'include' directive:
295
296 { 'include': 'path/to/file.json' }
297
298The directive is evaluated recursively, and include paths are relative to the
299file using the directive. Multiple includes of the same file are
300idempotent. No other keys should appear in the expression, and the include
301value should be a string.
302
303As a matter of style, it is a good idea to have all files be
304self-contained, but at the moment, nothing prevents an included file
305from making a forward reference to a type that is only introduced by
306an outer file. The parser may be made stricter in the future to
307prevent incomplete include files.
308
309
310=== Pragma directives ===
311
312Usage: { 'pragma': DICT }
313
314The pragma directive lets you control optional generator behavior.
315The dictionary's entries are pragma names and values.
316
317Pragma's scope is currently the complete schema. Setting the same
318pragma to different values in parts of the schema doesn't work.
319
320Pragma 'doc-required' takes a boolean value. If true, documentation
321is required. Default is false.
322
323Pragma 'returns-whitelist' takes a list of command names that may
324violate the rules on permitted return types. Default is none.
325
326Pragma 'name-case-whitelist' takes a list of names that may violate
327rules on use of upper- vs. lower-case letters. Default is none.
328
329
330=== Struct types ===
331
332Usage: { 'struct': STRING, 'data': DICT, '*base': STRUCT-NAME }
333
334A struct is a dictionary containing a single 'data' key whose value is
335a dictionary; the dictionary may be empty. This corresponds to a
336struct in C or an Object in JSON. Each value of the 'data' dictionary
337must be the name of a type, or a one-element array containing a type
338name. An example of a struct is:
339
340 { 'struct': 'MyType',
341 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
342
343The use of '*' as a prefix to the name means the member is optional in
344the corresponding JSON protocol usage.
345
346The default initialization value of an optional argument should not be changed
347between versions of QEMU unless the new default maintains backward
348compatibility to the user-visible behavior of the old default.
349
350With proper documentation, this policy still allows some flexibility; for
351example, documenting that a default of 0 picks an optimal buffer size allows
352one release to declare the optimal size at 512 while another release declares
353the optimal size at 4096 - the user-visible behavior is not the bytes used by
354the buffer, but the fact that the buffer was optimal size.
355
356On input structures (only mentioned in the 'data' side of a command), changing
357from mandatory to optional is safe (older clients will supply the option, and
358newer clients can benefit from the default); changing from optional to
359mandatory is backwards incompatible (older clients may be omitting the option,
360and must continue to work).
361
362On output structures (only mentioned in the 'returns' side of a command),
363changing from mandatory to optional is in general unsafe (older clients may be
364expecting the member, and could crash if it is missing), although it
365can be done if the only way that the optional argument will be omitted
366is when it is triggered by the presence of a new input flag to the
367command that older clients don't know to send. Changing from optional
368to mandatory is safe.
369
370A structure that is used in both input and output of various commands
371must consider the backwards compatibility constraints of both directions
372of use.
373
374A struct definition can specify another struct as its base.
375In this case, the members of the base type are included as top-level members
376of the new struct's dictionary in the Client JSON Protocol wire
377format. An example definition is:
378
379 { 'struct': 'BlockdevOptionsGenericFormat', 'data': { 'file': 'str' } }
380 { 'struct': 'BlockdevOptionsGenericCOWFormat',
381 'base': 'BlockdevOptionsGenericFormat',
382 'data': { '*backing': 'str' } }
383
384An example BlockdevOptionsGenericCOWFormat object on the wire could use
385both members like this:
386
387 { "file": "/some/place/my-image",
388 "backing": "/some/place/my-backing-file" }
389
390
391=== Enumeration types ===
392
393Usage: { 'enum': STRING, 'data': ARRAY-OF-STRING }
394 { 'enum': STRING, '*prefix': STRING, 'data': ARRAY-OF-STRING }
395
396An enumeration type is a dictionary containing a single 'data' key
397whose value is a list of strings. An example enumeration is:
398
399 { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
400
401Nothing prevents an empty enumeration, although it is probably not
402useful. The list of strings should be lower case; if an enum name
403represents multiple words, use '-' between words. The string 'max' is
404not allowed as an enum value, and values should not be repeated.
405
406The enum constants will be named by using a heuristic to turn the
407type name into a set of underscore separated words. For the example
408above, 'MyEnum' will turn into 'MY_ENUM' giving a constant name
409of 'MY_ENUM_VALUE1' for the first value. If the default heuristic
410does not result in a desirable name, the optional 'prefix' member
411can be used when defining the enum.
412
413The enumeration values are passed as strings over the Client JSON
414Protocol, but are encoded as C enum integral values in generated code.
415While the C code starts numbering at 0, it is better to use explicit
416comparisons to enum values than implicit comparisons to 0; the C code
417will also include a generated enum member ending in _MAX for tracking
418the size of the enum, useful when using common functions for
419converting between strings and enum values. Since the wire format
420always passes by name, it is acceptable to reorder or add new
421enumeration members in any location without breaking clients of Client
422JSON Protocol; however, removing enum values would break
423compatibility. For any struct that has a member that will only contain
424a finite set of string values, using an enum type for that member is
425better than open-coding the member to be type 'str'.
426
427
428=== Union types ===
429
430Usage: { 'union': STRING, 'data': DICT }
431or: { 'union': STRING, 'data': DICT, 'base': STRUCT-NAME-OR-DICT,
432 'discriminator': ENUM-MEMBER-OF-BASE }
433
434Union types are used to let the user choose between several different
435variants for an object. There are two flavors: simple (no
436discriminator or base), and flat (both discriminator and base). A union
437type is defined using a data dictionary as explained in the following
438paragraphs. The data dictionary for either type of union must not
439be empty.
440
441A simple union type defines a mapping from automatic discriminator
442values to data types like in this example:
443
444 { 'struct': 'BlockdevOptionsFile', 'data': { 'filename': 'str' } }
445 { 'struct': 'BlockdevOptionsQcow2',
446 'data': { 'backing': 'str', '*lazy-refcounts': 'bool' } }
447
448 { 'union': 'BlockdevOptionsSimple',
449 'data': { 'file': 'BlockdevOptionsFile',
450 'qcow2': 'BlockdevOptionsQcow2' } }
451
452In the Client JSON Protocol, a simple union is represented by a
453dictionary that contains the 'type' member as a discriminator, and a
454'data' member that is of the specified data type corresponding to the
455discriminator value, as in these examples:
456
457 { "type": "file", "data": { "filename": "/some/place/my-image" } }
458 { "type": "qcow2", "data": { "backing": "/some/place/my-image",
459 "lazy-refcounts": true } }
460
461The generated C code uses a struct containing a union. Additionally,
462an implicit C enum 'NameKind' is created, corresponding to the union
463'Name', for accessing the various branches of the union. No branch of
464the union can be named 'max', as this would collide with the implicit
465enum. The value for each branch can be of any type.
466
467A flat union definition avoids nesting on the wire, and specifies a
468set of common members that occur in all variants of the union. The
469'base' key must specify either a type name (the type must be a
470struct, not a union), or a dictionary representing an anonymous type.
471All branches of the union must be complex types, and the top-level
472members of the union dictionary on the wire will be combination of
473members from both the base type and the appropriate branch type (when
474merging two dictionaries, there must be no keys in common). The
475'discriminator' member must be the name of a non-optional enum-typed
476member of the base struct.
477
478The following example enhances the above simple union example by
479adding an optional common member 'read-only', renaming the
480discriminator to something more applicable than the simple union's
481default of 'type', and reducing the number of {} required on the wire:
482
483 { 'enum': 'BlockdevDriver', 'data': [ 'file', 'qcow2' ] }
484 { 'union': 'BlockdevOptions',
485 'base': { 'driver': 'BlockdevDriver', '*read-only': 'bool' },
486 'discriminator': 'driver',
487 'data': { 'file': 'BlockdevOptionsFile',
488 'qcow2': 'BlockdevOptionsQcow2' } }
489
490Resulting in these JSON objects:
491
492 { "driver": "file", "read-only": true,
493 "filename": "/some/place/my-image" }
494 { "driver": "qcow2", "read-only": false,
495 "backing": "/some/place/my-image", "lazy-refcounts": true }
496
497Notice that in a flat union, the discriminator name is controlled by
498the user, but because it must map to a base member with enum type, the
499code generator can ensure that branches exist for all values of the
500enum (although the order of the keys need not match the declaration of
501the enum). In the resulting generated C data types, a flat union is
502represented as a struct with the base members included directly, and
503then a union of structures for each branch of the struct.
504
505A simple union can always be re-written as a flat union where the base
506class has a single member named 'type', and where each branch of the
507union has a struct with a single member named 'data'. That is,
508
509 { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
510
511is identical on the wire to:
512
513 { 'enum': 'Enum', 'data': ['one', 'two'] }
514 { 'struct': 'Branch1', 'data': { 'data': 'str' } }
515 { 'struct': 'Branch2', 'data': { 'data': 'int' } }
516 { 'union': 'Flat': 'base': { 'type': 'Enum' }, 'discriminator': 'type',
517 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
518
519
520=== Alternate types ===
521
522Usage: { 'alternate': STRING, 'data': DICT }
523
524An alternate type is one that allows a choice between two or more JSON
525data types (string, integer, number, or object, but currently not
526array) on the wire. The definition is similar to a simple union type,
527where each branch of the union names a QAPI type. For example:
528
529 { 'alternate': 'BlockdevRef',
530 'data': { 'definition': 'BlockdevOptions',
531 'reference': 'str' } }
532
533Unlike a union, the discriminator string is never passed on the wire
534for the Client JSON Protocol. Instead, the value's JSON type serves
535as an implicit discriminator, which in turn means that an alternate
536can only express a choice between types represented differently in
537JSON. If a branch is typed as the 'bool' built-in, the alternate
538accepts true and false; if it is typed as any of the various numeric
539built-ins, it accepts a JSON number; if it is typed as a 'str'
540built-in or named enum type, it accepts a JSON string; if it is typed
541as the 'null' built-in, it accepts JSON null; and if it is typed as a
542complex type (struct or union), it accepts a JSON object. Two
543different complex types, for instance, aren't permitted, because both
544are represented as a JSON object.
545
546The example alternate declaration above allows using both of the
547following example objects:
548
549 { "file": "my_existing_block_device_id" }
550 { "file": { "driver": "file",
551 "read-only": false,
552 "filename": "/tmp/mydisk.qcow2" } }
553
554
555=== Commands ===
556
557--- General Command Layout ---
558
559Usage: { 'command': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,
560 '*returns': TYPE-NAME, '*boxed': true,
561 '*gen': false, '*success-response': false,
562 '*allow-oob': true }
563
564Commands are defined by using a dictionary containing several members,
565where three members are most common. The 'command' member is a
566mandatory string, and determines the "execute" value passed in a
567Client JSON Protocol command exchange.
568
569The 'data' argument maps to the "arguments" dictionary passed in as
570part of a Client JSON Protocol command. The 'data' member is optional
571and defaults to {} (an empty dictionary). If present, it must be the
572string name of a complex type, or a dictionary that declares an
573anonymous type with the same semantics as a 'struct' expression.
574
575The 'returns' member describes what will appear in the "return" member
576of a Client JSON Protocol reply on successful completion of a command.
577The member is optional from the command declaration; if absent, the
578"return" member will be an empty dictionary. If 'returns' is present,
579it must be the string name of a complex or built-in type, a
580one-element array containing the name of a complex or built-in type.
581To return anything else, you have to list the command in pragma
582'returns-whitelist'. If you do this, the command cannot be extended
583to return additional information in the future. Use of
584'returns-whitelist' for new commands is strongly discouraged.
585
586All commands in Client JSON Protocol use a dictionary to report
587failure, with no way to specify that in QAPI. Where the error return
588is different than the usual GenericError class in order to help the
589client react differently to certain error conditions, it is worth
590documenting this in the comments before the command declaration.
591
592Some example commands:
593
594 { 'command': 'my-first-command',
595 'data': { 'arg1': 'str', '*arg2': 'str' } }
596 { 'struct': 'MyType', 'data': { '*value': 'str' } }
597 { 'command': 'my-second-command',
598 'returns': [ 'MyType' ] }
599
600which would validate this Client JSON Protocol transaction:
601
602 => { "execute": "my-first-command",
603 "arguments": { "arg1": "hello" } }
604 <= { "return": { } }
605 => { "execute": "my-second-command" }
606 <= { "return": [ { "value": "one" }, { } ] }
607
608The generator emits a prototype for the user's function implementing
609the command. Normally, 'data' is a dictionary for an anonymous type,
610or names a struct type (possibly empty, but not a union), and its
611members are passed as separate arguments to this function. If the
612command definition includes a key 'boxed' with the boolean value true,
613then 'data' is instead the name of any non-empty complex type
614(struct, union, or alternate), and a pointer to that QAPI type is
615passed as a single argument.
616
617The generator also emits a marshalling function that extracts
618arguments for the user's function out of an input QDict, calls the
619user's function, and if it succeeded, builds an output QObject from
620its return value.
621
622In rare cases, QAPI cannot express a type-safe representation of a
623corresponding Client JSON Protocol command. You then have to suppress
624generation of a marshalling function by including a key 'gen' with
625boolean value false, and instead write your own function. Please try
626to avoid adding new commands that rely on this, and instead use
627type-safe unions. For an example of this usage:
628
629 { 'command': 'netdev_add',
630 'data': {'type': 'str', 'id': 'str'},
631 'gen': false }
632
633Normally, the QAPI schema is used to describe synchronous exchanges,
634where a response is expected. But in some cases, the action of a
635command is expected to change state in a way that a successful
636response is not possible (although the command will still return a
637normal dictionary error on failure). When a successful reply is not
638possible, the command expression should include the optional key
639'success-response' with boolean value false. So far, only QGA makes
640use of this member.
641
642A command can be declared to support Out-Of-Band (OOB) execution. By
643default, commands do not support OOB. To declare a command that
644supports it, the schema includes an extra 'allow-oob' field. For
645example:
646
647 { 'command': 'migrate_recover',
648 'data': { 'uri': 'str' }, 'allow-oob': true }
649
650To execute a command with out-of-band priority, the client specifies
651the "control" field in the request, with "run-oob" set to
652true. Example:
653
654 => { "execute": "command-support-oob",
655 "arguments": { ... },
656 "control": { "run-oob": true } }
657 <= { "return": { } }
658
659Without it, even the commands that support out-of-band execution will
660still be run in-band.
661
662Under normal QMP command execution, the following apply to each
663command:
664
665- They are executed in order,
666- They run only in main thread of QEMU,
667- They have the BQL taken during execution.
668
669When a command is executed with OOB, the following changes occur:
670
671- They can be completed before a pending in-band command,
672- They run in a dedicated monitor thread,
673- They do not take the BQL during execution.
674
675OOB command handlers must satisfy the following conditions:
676
677- It executes extremely fast,
678- It does not take any lock, or, it can take very small locks if all
679 critical regions also follow the rules for OOB command handler code,
680- It does not invoke system calls that may block,
681- It does not access guest RAM that may block when userfaultfd is
682 enabled for postcopy live migration.
683
684If in doubt, do not implement OOB execution support.
685
686=== Events ===
687
688Usage: { 'event': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,
689 '*boxed': true }
690
691Events are defined with the keyword 'event'. It is not allowed to
692name an event 'MAX', since the generator also produces a C enumeration
693of all event names with a generated _MAX value at the end. When
694'data' is also specified, additional info will be included in the
695event, with similar semantics to a 'struct' expression. Finally there
696will be C API generated in qapi-events.h; when called by QEMU code, a
697message with timestamp will be emitted on the wire.
698
699An example event is:
700
701{ 'event': 'EVENT_C',
702 'data': { '*a': 'int', 'b': 'str' } }
703
704Resulting in this JSON object:
705
706{ "event": "EVENT_C",
707 "data": { "b": "test string" },
708 "timestamp": { "seconds": 1267020223, "microseconds": 435656 } }
709
710The generator emits a function to send the event. Normally, 'data' is
711a dictionary for an anonymous type, or names a struct type (possibly
712empty, but not a union), and its members are passed as separate
713arguments to this function. If the event definition includes a key
714'boxed' with the boolean value true, then 'data' is instead the name of
715any non-empty complex type (struct, union, or alternate), and a
716pointer to that QAPI type is passed as a single argument.
717
718
719=== Downstream extensions ===
720
721QAPI schema names that are externally visible, say in the Client JSON
722Protocol, need to be managed with care. Names starting with a
723downstream prefix of the form __RFQDN_ are reserved for the downstream
724who controls the valid, reverse fully qualified domain name RFQDN.
725RFQDN may only contain ASCII letters, digits, hyphen and period.
726
727Example: Red Hat, Inc. controls redhat.com, and may therefore add a
728downstream command __com.redhat_drive-mirror.
729
730
731== Client JSON Protocol introspection ==
732
733Clients of a Client JSON Protocol commonly need to figure out what
734exactly the server (QEMU) supports.
735
736For this purpose, QMP provides introspection via command
737query-qmp-schema. QGA currently doesn't support introspection.
738
739While Client JSON Protocol wire compatibility should be maintained
740between qemu versions, we cannot make the same guarantees for
741introspection stability. For example, one version of qemu may provide
742a non-variant optional member of a struct, and a later version rework
743the member to instead be non-optional and associated with a variant.
744Likewise, one version of qemu may list a member with open-ended type
745'str', and a later version could convert it to a finite set of strings
746via an enum type; or a member may be converted from a specific type to
747an alternate that represents a choice between the original type and
748something else.
749
750query-qmp-schema returns a JSON array of SchemaInfo objects. These
751objects together describe the wire ABI, as defined in the QAPI schema.
752There is no specified order to the SchemaInfo objects returned; a
753client must search for a particular name throughout the entire array
754to learn more about that name, but is at least guaranteed that there
755will be no collisions between type, command, and event names.
756
757However, the SchemaInfo can't reflect all the rules and restrictions
758that apply to QMP. It's interface introspection (figuring out what's
759there), not interface specification. The specification is in the QAPI
760schema. To understand how QMP is to be used, you need to study the
761QAPI schema.
762
763Like any other command, query-qmp-schema is itself defined in the QAPI
764schema, along with the SchemaInfo type. This text attempts to give an
765overview how things work. For details you need to consult the QAPI
766schema.
767
768SchemaInfo objects have common members "name" and "meta-type", and
769additional variant members depending on the value of meta-type.
770
771Each SchemaInfo object describes a wire ABI entity of a certain
772meta-type: a command, event or one of several kinds of type.
773
774SchemaInfo for commands and events have the same name as in the QAPI
775schema.
776
777Command and event names are part of the wire ABI, but type names are
778not. Therefore, the SchemaInfo for types have auto-generated
779meaningless names. For readability, the examples in this section use
780meaningful type names instead.
781
782To examine a type, start with a command or event using it, then follow
783references by name.
784
785QAPI schema definitions not reachable that way are omitted.
786
787The SchemaInfo for a command has meta-type "command", and variant
788members "arg-type", "ret-type" and "allow-oob". On the wire, the
789"arguments" member of a client's "execute" command must conform to the
790object type named by "arg-type". The "return" member that the server
791passes in a success response conforms to the type named by
792"ret-type". When "allow-oob" is set, it means the command supports
793out-of-band execution.
794
795If the command takes no arguments, "arg-type" names an object type
796without members. Likewise, if the command returns nothing, "ret-type"
797names an object type without members.
798
799Example: the SchemaInfo for command query-qmp-schema
800
801 { "name": "query-qmp-schema", "meta-type": "command",
802 "arg-type": "q_empty", "ret-type": "SchemaInfoList" }
803
804 Type "q_empty" is an automatic object type without members, and type
805 "SchemaInfoList" is the array of SchemaInfo type.
806
807The SchemaInfo for an event has meta-type "event", and variant member
808"arg-type". On the wire, a "data" member that the server passes in an
809event conforms to the object type named by "arg-type".
810
811If the event carries no additional information, "arg-type" names an
812object type without members. The event may not have a data member on
813the wire then.
814
815Each command or event defined with dictionary-valued 'data' in the
816QAPI schema implicitly defines an object type.
817
818Example: the SchemaInfo for EVENT_C from section Events
819
820 { "name": "EVENT_C", "meta-type": "event",
821 "arg-type": "q_obj-EVENT_C-arg" }
822
823 Type "q_obj-EVENT_C-arg" is an implicitly defined object type with
824 the two members from the event's definition.
825
826The SchemaInfo for struct and union types has meta-type "object".
827
828The SchemaInfo for a struct type has variant member "members".
829
830The SchemaInfo for a union type additionally has variant members "tag"
831and "variants".
832
833"members" is a JSON array describing the object's common members, if
834any. Each element is a JSON object with members "name" (the member's
835name), "type" (the name of its type), and optionally "default". The
836member is optional if "default" is present. Currently, "default" can
837only have value null. Other values are reserved for future
838extensions. The "members" array is in no particular order; clients
839must search the entire object when learning whether a particular
840member is supported.
841
842Example: the SchemaInfo for MyType from section Struct types
843
844 { "name": "MyType", "meta-type": "object",
845 "members": [
846 { "name": "member1", "type": "str" },
847 { "name": "member2", "type": "int" },
848 { "name": "member3", "type": "str", "default": null } ] }
849
850"tag" is the name of the common member serving as type tag.
851"variants" is a JSON array describing the object's variant members.
852Each element is a JSON object with members "case" (the value of type
853tag this element applies to) and "type" (the name of an object type
854that provides the variant members for this type tag value). The
855"variants" array is in no particular order, and is not guaranteed to
856list cases in the same order as the corresponding "tag" enum type.
857
858Example: the SchemaInfo for flat union BlockdevOptions from section
859Union types
860
861 { "name": "BlockdevOptions", "meta-type": "object",
862 "members": [
863 { "name": "driver", "type": "BlockdevDriver" },
864 { "name": "read-only", "type": "bool", "default": null } ],
865 "tag": "driver",
866 "variants": [
867 { "case": "file", "type": "BlockdevOptionsFile" },
868 { "case": "qcow2", "type": "BlockdevOptionsQcow2" } ] }
869
870Note that base types are "flattened": its members are included in the
871"members" array.
872
873A simple union implicitly defines an enumeration type for its implicit
874discriminator (called "type" on the wire, see section Union types).
875
876A simple union implicitly defines an object type for each of its
877variants.
878
879Example: the SchemaInfo for simple union BlockdevOptionsSimple from section
880Union types
881
882 { "name": "BlockdevOptionsSimple", "meta-type": "object",
883 "members": [
884 { "name": "type", "type": "BlockdevOptionsSimpleKind" } ],
885 "tag": "type",
886 "variants": [
887 { "case": "file", "type": "q_obj-BlockdevOptionsFile-wrapper" },
888 { "case": "qcow2", "type": "q_obj-BlockdevOptionsQcow2-wrapper" } ] }
889
890 Enumeration type "BlockdevOptionsSimpleKind" and the object types
891 "q_obj-BlockdevOptionsFile-wrapper", "q_obj-BlockdevOptionsQcow2-wrapper"
892 are implicitly defined.
893
894The SchemaInfo for an alternate type has meta-type "alternate", and
895variant member "members". "members" is a JSON array. Each element is
896a JSON object with member "type", which names a type. Values of the
897alternate type conform to exactly one of its member types. There is
898no guarantee on the order in which "members" will be listed.
899
900Example: the SchemaInfo for BlockdevRef from section Alternate types
901
902 { "name": "BlockdevRef", "meta-type": "alternate",
903 "members": [
904 { "type": "BlockdevOptions" },
905 { "type": "str" } ] }
906
907The SchemaInfo for an array type has meta-type "array", and variant
908member "element-type", which names the array's element type. Array
909types are implicitly defined. For convenience, the array's name may
910resemble the element type; however, clients should examine member
911"element-type" instead of making assumptions based on parsing member
912"name".
913
914Example: the SchemaInfo for ['str']
915
916 { "name": "[str]", "meta-type": "array",
917 "element-type": "str" }
918
919The SchemaInfo for an enumeration type has meta-type "enum" and
920variant member "values". The values are listed in no particular
921order; clients must search the entire enum when learning whether a
922particular value is supported.
923
924Example: the SchemaInfo for MyEnum from section Enumeration types
925
926 { "name": "MyEnum", "meta-type": "enum",
927 "values": [ "value1", "value2", "value3" ] }
928
929The SchemaInfo for a built-in type has the same name as the type in
930the QAPI schema (see section Built-in Types), with one exception
931detailed below. It has variant member "json-type" that shows how
932values of this type are encoded on the wire.
933
934Example: the SchemaInfo for str
935
936 { "name": "str", "meta-type": "builtin", "json-type": "string" }
937
938The QAPI schema supports a number of integer types that only differ in
939how they map to C. They are identical as far as SchemaInfo is
940concerned. Therefore, they get all mapped to a single type "int" in
941SchemaInfo.
942
943As explained above, type names are not part of the wire ABI. Not even
944the names of built-in types. Clients should examine member
945"json-type" instead of hard-coding names of built-in types.
946
947
948== Code generation ==
949
950The QAPI code generator qapi-gen.py generates code and documentation
951from the schema. Together with the core QAPI libraries, this code
952provides everything required to take JSON commands read in by a Client
953JSON Protocol server, unmarshal the arguments into the underlying C
954types, call into the corresponding C function, map the response back
955to a Client JSON Protocol response to be returned to the user, and
956introspect the commands.
957
958As an example, we'll use the following schema, which describes a
959single complex user-defined type, along with command which takes a
960list of that type as a parameter, and returns a single element of that
961type. The user is responsible for writing the implementation of
962qmp_my_command(); everything else is produced by the generator.
963
964 $ cat example-schema.json
965 { 'struct': 'UserDefOne',
966 'data': { 'integer': 'int', '*string': 'str' } }
967
968 { 'command': 'my-command',
969 'data': { 'arg1': ['UserDefOne'] },
970 'returns': 'UserDefOne' }
971
972 { 'event': 'MY_EVENT' }
973
974We run qapi-gen.py like this:
975
976 $ python scripts/qapi-gen.py --output-dir="qapi-generated" \
977 --prefix="example-" example-schema.json
978
979For a more thorough look at generated code, the testsuite includes
980tests/qapi-schema/qapi-schema-tests.json that covers more examples of
981what the generator will accept, and compiles the resulting C code as
982part of 'make check-unit'.
983
984=== Code generated for QAPI types ===
985
986The following files are created:
987
988$(prefix)qapi-types.h - C types corresponding to types defined in
989 the schema
990
991$(prefix)qapi-types.c - Cleanup functions for the above C types
992
993The $(prefix) is an optional parameter used as a namespace to keep the
994generated code from one schema/code-generation separated from others so code
995can be generated/used from multiple schemas without clobbering previously
996created code.
997
998Example:
999
1000 $ cat qapi-generated/example-qapi-types.h
1001[Uninteresting stuff omitted...]
1002
1003 #ifndef EXAMPLE_QAPI_TYPES_H
1004 #define EXAMPLE_QAPI_TYPES_H
1005
1006[Built-in types omitted...]
1007
1008 typedef struct UserDefOne UserDefOne;
1009
1010 typedef struct UserDefOneList UserDefOneList;
1011
1012 typedef struct q_obj_my_command_arg q_obj_my_command_arg;
1013
1014 struct UserDefOne {
1015 int64_t integer;
1016 bool has_string;
1017 char *string;
1018 };
1019
1020 void qapi_free_UserDefOne(UserDefOne *obj);
1021
1022 struct UserDefOneList {
1023 UserDefOneList *next;
1024 UserDefOne *value;
1025 };
1026
1027 void qapi_free_UserDefOneList(UserDefOneList *obj);
1028
1029 struct q_obj_my_command_arg {
1030 UserDefOneList *arg1;
1031 };
1032
1033 #endif
1034 $ cat qapi-generated/example-qapi-types.c
1035[Uninteresting stuff omitted...]
1036
1037 void qapi_free_UserDefOne(UserDefOne *obj)
1038 {
1039 Visitor *v;
1040
1041 if (!obj) {
1042 return;
1043 }
1044
1045 v = qapi_dealloc_visitor_new();
1046 visit_type_UserDefOne(v, NULL, &obj, NULL);
1047 visit_free(v);
1048 }
1049
1050 void qapi_free_UserDefOneList(UserDefOneList *obj)
1051 {
1052 Visitor *v;
1053
1054 if (!obj) {
1055 return;
1056 }
1057
1058 v = qapi_dealloc_visitor_new();
1059 visit_type_UserDefOneList(v, NULL, &obj, NULL);
1060 visit_free(v);
1061 }
1062
1063=== Code generated for visiting QAPI types ===
1064
1065These are the visitor functions used to walk through and convert
1066between a native QAPI C data structure and some other format (such as
1067QObject); the generated functions are named visit_type_FOO() and
1068visit_type_FOO_members().
1069
1070The following files are generated:
1071
1072$(prefix)qapi-visit.c: Visitor function for a particular C type, used
1073 to automagically convert QObjects into the
1074 corresponding C type and vice-versa, as well
1075 as for deallocating memory for an existing C
1076 type
1077
1078$(prefix)qapi-visit.h: Declarations for previously mentioned visitor
1079 functions
1080
1081Example:
1082
1083 $ cat qapi-generated/example-qapi-visit.h
1084[Uninteresting stuff omitted...]
1085
1086 #ifndef EXAMPLE_QAPI_VISIT_H
1087 #define EXAMPLE_QAPI_VISIT_H
1088
1089[Visitors for built-in types omitted...]
1090
1091 void visit_type_UserDefOne_members(Visitor *v, UserDefOne *obj, Error **errp);
1092 void visit_type_UserDefOne(Visitor *v, const char *name, UserDefOne **obj, Error **errp);
1093 void visit_type_UserDefOneList(Visitor *v, const char *name, UserDefOneList **obj, Error **errp);
1094
1095 void visit_type_q_obj_my_command_arg_members(Visitor *v, q_obj_my_command_arg *obj, Error **errp);
1096
1097 #endif
1098 $ cat qapi-generated/example-qapi-visit.c
1099[Uninteresting stuff omitted...]
1100
1101 void visit_type_UserDefOne_members(Visitor *v, UserDefOne *obj, Error **errp)
1102 {
1103 Error *err = NULL;
1104
1105 visit_type_int(v, "integer", &obj->integer, &err);
1106 if (err) {
1107 goto out;
1108 }
1109 if (visit_optional(v, "string", &obj->has_string)) {
1110 visit_type_str(v, "string", &obj->string, &err);
1111 if (err) {
1112 goto out;
1113 }
1114 }
1115
1116 out:
1117 error_propagate(errp, err);
1118 }
1119
1120 void visit_type_UserDefOne(Visitor *v, const char *name, UserDefOne **obj, Error **errp)
1121 {
1122 Error *err = NULL;
1123
1124 visit_start_struct(v, name, (void **)obj, sizeof(UserDefOne), &err);
1125 if (err) {
1126 goto out;
1127 }
1128 if (!*obj) {
1129 goto out_obj;
1130 }
1131 visit_type_UserDefOne_members(v, *obj, &err);
1132 if (err) {
1133 goto out_obj;
1134 }
1135 visit_check_struct(v, &err);
1136 out_obj:
1137 visit_end_struct(v, (void **)obj);
1138 if (err && visit_is_input(v)) {
1139 qapi_free_UserDefOne(*obj);
1140 *obj = NULL;
1141 }
1142 out:
1143 error_propagate(errp, err);
1144 }
1145
1146 void visit_type_UserDefOneList(Visitor *v, const char *name, UserDefOneList **obj, Error **errp)
1147 {
1148 Error *err = NULL;
1149 UserDefOneList *tail;
1150 size_t size = sizeof(**obj);
1151
1152 visit_start_list(v, name, (GenericList **)obj, size, &err);
1153 if (err) {
1154 goto out;
1155 }
1156
1157 for (tail = *obj; tail;
1158 tail = (UserDefOneList *)visit_next_list(v, (GenericList *)tail, size)) {
1159 visit_type_UserDefOne(v, NULL, &tail->value, &err);
1160 if (err) {
1161 break;
1162 }
1163 }
1164
1165 if (!err) {
1166 visit_check_list(v, &err);
1167 }
1168 visit_end_list(v, (void **)obj);
1169 if (err && visit_is_input(v)) {
1170 qapi_free_UserDefOneList(*obj);
1171 *obj = NULL;
1172 }
1173 out:
1174 error_propagate(errp, err);
1175 }
1176
1177 void visit_type_q_obj_my_command_arg_members(Visitor *v, q_obj_my_command_arg *obj, Error **errp)
1178 {
1179 Error *err = NULL;
1180
1181 visit_type_UserDefOneList(v, "arg1", &obj->arg1, &err);
1182 if (err) {
1183 goto out;
1184 }
1185
1186 out:
1187 error_propagate(errp, err);
1188 }
1189
1190=== Code generated for commands ===
1191
1192These are the marshaling/dispatch functions for the commands defined
1193in the schema. The generated code provides qmp_marshal_COMMAND(), and
1194declares qmp_COMMAND() that the user must implement.
1195
1196The following files are generated:
1197
1198$(prefix)qapi-commands.c: Command marshal/dispatch functions for each
1199 QMP command defined in the schema
1200
1201$(prefix)qapi-commands.h: Function prototypes for the QMP commands
1202 specified in the schema
1203
1204Example:
1205
1206 $ cat qapi-generated/example-qapi-commands.h
1207[Uninteresting stuff omitted...]
1208
1209 #ifndef EXAMPLE_QMP_COMMANDS_H
1210 #define EXAMPLE_QMP_COMMANDS_H
1211
1212 #include "example-qapi-types.h"
1213 #include "qapi/qmp/qdict.h"
1214 #include "qapi/qmp/dispatch.h"
1215
1216 void example_qmp_init_marshal(QmpCommandList *cmds);
1217 UserDefOne *qmp_my_command(UserDefOneList *arg1, Error **errp);
1218 void qmp_marshal_my_command(QDict *args, QObject **ret, Error **errp);
1219
1220 #endif
1221 $ cat qapi-generated/example-qapi-commands.c
1222[Uninteresting stuff omitted...]
1223
1224 static void qmp_marshal_output_UserDefOne(UserDefOne *ret_in, QObject **ret_out, Error **errp)
1225 {
1226 Error *err = NULL;
1227 Visitor *v;
1228
1229 v = qobject_output_visitor_new(ret_out);
1230 visit_type_UserDefOne(v, "unused", &ret_in, &err);
1231 if (!err) {
1232 visit_complete(v, ret_out);
1233 }
1234 error_propagate(errp, err);
1235 visit_free(v);
1236 v = qapi_dealloc_visitor_new();
1237 visit_type_UserDefOne(v, "unused", &ret_in, NULL);
1238 visit_free(v);
1239 }
1240
1241 void qmp_marshal_my_command(QDict *args, QObject **ret, Error **errp)
1242 {
1243 Error *err = NULL;
1244 UserDefOne *retval;
1245 Visitor *v;
1246 q_obj_my_command_arg arg = {0};
1247
1248 v = qobject_input_visitor_new(QOBJECT(args));
1249 visit_start_struct(v, NULL, NULL, 0, &err);
1250 if (err) {
1251 goto out;
1252 }
1253 visit_type_q_obj_my_command_arg_members(v, &arg, &err);
1254 if (!err) {
1255 visit_check_struct(v, &err);
1256 }
1257 visit_end_struct(v, NULL);
1258 if (err) {
1259 goto out;
1260 }
1261
1262 retval = qmp_my_command(arg.arg1, &err);
1263 if (err) {
1264 goto out;
1265 }
1266
1267 qmp_marshal_output_UserDefOne(retval, ret, &err);
1268
1269 out:
1270 error_propagate(errp, err);
1271 visit_free(v);
1272 v = qapi_dealloc_visitor_new();
1273 visit_start_struct(v, NULL, NULL, 0, NULL);
1274 visit_type_q_obj_my_command_arg_members(v, &arg, NULL);
1275 visit_end_struct(v, NULL);
1276 visit_free(v);
1277 }
1278
1279 void example_qmp_init_marshal(QmpCommandList *cmds)
1280 {
1281 QTAILQ_INIT(cmds);
1282
1283 qmp_register_command(cmds, "my-command",
1284 qmp_marshal_my_command, QCO_NO_OPTIONS);
1285 }
1286
1287=== Code generated for events ===
1288
1289This is the code related to events defined in the schema, providing
1290qapi_event_send_EVENT().
1291
1292The following files are created:
1293
1294$(prefix)qapi-events.h - Function prototypes for each event type, plus an
1295 enumeration of all event names
1296
1297$(prefix)qapi-events.c - Implementation of functions to send an event
1298
1299Example:
1300
1301 $ cat qapi-generated/example-qapi-events.h
1302[Uninteresting stuff omitted...]
1303
1304 #ifndef EXAMPLE_QAPI_EVENT_H
1305 #define EXAMPLE_QAPI_EVENT_H
1306
1307 #include "qapi/qmp/qdict.h"
1308 #include "example-qapi-types.h"
1309
1310
1311 void qapi_event_send_my_event(Error **errp);
1312
1313 typedef enum example_QAPIEvent {
1314 EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
1315 EXAMPLE_QAPI_EVENT__MAX = 1,
1316 } example_QAPIEvent;
1317
1318 #define example_QAPIEvent_str(val) \
1319 qapi_enum_lookup(example_QAPIEvent_lookup, (val))
1320
1321 extern const char *const example_QAPIEvent_lookup[];
1322
1323 #endif
1324 $ cat qapi-generated/example-qapi-events.c
1325[Uninteresting stuff omitted...]
1326
1327 void qapi_event_send_my_event(Error **errp)
1328 {
1329 QDict *qmp;
1330 Error *err = NULL;
1331 QMPEventFuncEmit emit;
1332
1333 emit = qmp_event_get_func_emit();
1334 if (!emit) {
1335 return;
1336 }
1337
1338 qmp = qmp_event_build_dict("MY_EVENT");
1339
1340 emit(EXAMPLE_QAPI_EVENT_MY_EVENT, qmp, &err);
1341
1342 error_propagate(errp, err);
1343 QDECREF(qmp);
1344 }
1345
1346 const QEnumLookup example_QAPIEvent_lookup = {
1347 .array = (const char *const[]) {
1348 [EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
1349 },
1350 .size = EXAMPLE_QAPI_EVENT__MAX
1351 };
1352
1353=== Code generated for introspection ===
1354
1355The following files are created:
1356
1357$(prefix)qapi-introspect.c - Defines a string holding a JSON
1358 description of the schema
1359
1360$(prefix)qapi-introspect.h - Declares the above string
1361
1362Example:
1363
1364 $ cat qapi-generated/example-qapi-introspect.h
1365[Uninteresting stuff omitted...]
1366
1367 #ifndef EXAMPLE_QMP_INTROSPECT_H
1368 #define EXAMPLE_QMP_INTROSPECT_H
1369
1370 extern const QLitObject qmp_schema_qlit;
1371
1372 #endif
1373 $ cat qapi-generated/example-qapi-introspect.c
1374[Uninteresting stuff omitted...]
1375
1376 const QLitObject example_qmp_schema_qlit = QLIT_QLIST(((QLitObject[]) {
1377 QLIT_QDICT(((QLitDictEntry[]) {
1378 { "arg-type", QLIT_QSTR("0") },
1379 { "meta-type", QLIT_QSTR("event") },
1380 { "name", QLIT_QSTR("Event") },
1381 { }
1382 })),
1383 QLIT_QDICT(((QLitDictEntry[]) {
1384 { "members", QLIT_QLIST(((QLitObject[]) {
1385 { }
1386 })) },
1387 { "meta-type", QLIT_QSTR("object") },
1388 { "name", QLIT_QSTR("0") },
1389 { }
1390 })),
1391 ...
1392 { }
1393 }));