The library is implemented in C++. The source files have the .c
extension for legacy reasons.
libgccjit.cc implements the API entrypoints. It performs error
checking, then calls into classes of the gcc::jit::recording namespace
within jit-recording.cc and jit-recording.h.
jit-recording.cc and
jit-recording.h) record the API calls that are made:
/* Indentation indicates inheritance: */ class context; class memento; class string; class location; class type; class function_type; class compound_type; class struct_; class union_; class vector_type; class field; class bitfield; class fields; class function; class block; class rvalue; class lvalue; class local; class global; class param; class base_call; class function_pointer; class statement; class extended_asm; class case_; class top_level_asm;
jit-playback.cc and jit-playback.h) replay the API calls
within langhook:parse_file:
/* Indentation indicates inheritance: */ class context; class wrapper; class type; class compound_type; class field; class function; class block; class rvalue; class lvalue; class param; class source_file; class source_line; class location; class case_;Client Code . Generated . libgccjit.so . code . . . JIT API . JIT "Frontend". (libbackend.a) .................................................................................... │ . . . . ──────────────────────────> . . . . │ . . . . V . . . . ──> libgccjit.cc . . . │ (error-checking). . . │ . . . ──> jit-recording.cc . . (record API calls) . . <─────── . . . │ . . <─────────────────────────── . . │ . . . . │ . . . . V . . gcc_jit_context_compile . ──────────────────────────> . . . . │ start of recording::context::compile () . . │ . . . . │ start of playback::context::compile () . . │ (create tempdir) . . . │ . . . . │ ACQUIRE MUTEX . . . │ . . . . V───────────────────────> toplev::main (for now) . . . . │ . . . . (various code) . . . . │ . . . . V . . . <───────────────── langhook:parse_file . . . │ . . . . │ (jit_langhook_parse_file) . . . │ . ..........................................│..................VVVVVVVVVVVVV... . . . │ . No GC in here . . . │ jit-playback.cc . . . │ (playback of API calls) . . . ───────────────> creation of functions, . . . . types, expression trees . . . <──────────────── etc . . . │(handle_locations: add locations to . . . │ linemap and associate them with trees) . . . │ . . . . │ . No GC in here ..........................................│..................AAAAAAAAAAAAA... . . . │ for each function . . . ──> postprocess . . . │ . . . . ────────────> cgraph_finalize_function . . . <──────────── . . . <── . . . . │ . . . . ──────────────────> (end of . . . . │ langhook_parse_file) . . . . │ . . . . (various code) . . . . │ . . . . ↓ . . . <───────────────── langhook:write_globals . . . │ . . . . │ (jit_langhook_write_globals) . . . │ . . . . │ . . . . ──────────────────> finalize_compilation_unit . . . . │ . . . . (the middle─end and backend) . . . . ↓ . . <───────────────────────────── end of toplev::main . . │ . . . . V───────────────────────> toplev::finalize . . . . │ (purge internal state) . . <──────────────────────── end of toplev::finalize . . │ . . . . V─> playback::context::postprocess: . . │ . . . . │ (assuming an in-memory compile): . . │ . . . . --> Convert assembler to DSO, via embedded . . copy of driver: . . driver::main () . . invocation of "as" . . invocation of "ld" . . driver::finalize () . . <---- . . │ . . . . │ . Load DSO (dlopen "fake.so") . . │ . . . . │ . Bundle it up in a jit::result . . <── . . . . │ . . . . │ RELEASE MUTEX . . . │ . . . . │ end of playback::context::compile () . . │ . . . . │ playback::context dtor . . ──> . . . . │ Normally we cleanup the tempdir here: . . │ ("fake.so" is unlinked from the . . │ filesystem at this point) . . │ If the client code requested debuginfo, the . . │ cleanup happens later (in gcc_jit_result_release) . . │ to make it easier on the debugger (see PR jit/64206) . . <── . . . . │ . . . . │ end of recording::context::compile () <─────────────────────────── . . │ . . . . V . . gcc_jit_result_get_code . ──────────────────────────> . . . . │ dlsym () within loaded DSO <─────────────────────────── . . Get (void*). . . . │ . . . . │ Call it . . . . ───────────────> . . . . │ . . . . │ . . . <─────────────── . . . │ . . . . etc│ . . . . │ . . . . V . . gcc_jit_result_release . ──────────────────────────> . . . . │ dlclose () the loaded DSO . . │ (code becomes uncallable) . . │ . . . . │ If the client code requested debuginfo, then . . │ cleanup of the tempdir was delayed. . . │ If that was the case, clean it up now. <─────────────────────────── . . │ . . . .
Here is a high-level summary from jit-common.h:
In order to allow jit objects to be usable outside of a compile whilst working with the existing structure of GCC’s code the C API is implemented in terms of a gcc::jit::recording::context, which records the calls made to it.
When a gcc_jit_context is compiled, the recording context creates a playback context. The playback context invokes the bulk of the GCC code, and within the “frontend” parsing hook, plays back the recorded API calls, creating GCC tree objects.
So there are two parallel families of classes: those relating to recording, and those relating to playback:
- Visibility: recording objects are exposed back to client code, whereas playback objects are internal to the library.
- Lifetime: recording objects have a lifetime equal to that of the recording context that created them, whereas playback objects only exist within the frontend hook.
- Memory allocation: recording objects are allocated by the recording context, and automatically freed by it when the context is released, whereas playback objects are allocated within the GC heap, and garbage-collected; they can own GC-references.
- Integration with rest of GCC: recording objects are unrelated to the rest of GCC, whereas playback objects are wrappers around “tree” instances. Hence you can’t ask a recording rvalue or lvalue what its type is, whereas you can for a playback rvalue of lvalue (since it can work with the underlying GCC tree nodes).
- Instancing: There can be multiple recording contexts “alive” at once (albeit it only one compiling at once), whereas there can only be one playback context alive at one time (since it interacts with the GC).
Ultimately if GCC could support multiple GC heaps and contexts, and finer-grained initialization, then this recording vs playback distinction could be eliminated.
During a playback, we associate objects from the recording with their counterparts during this playback. For simplicity, we store this within the recording objects, as
void *m_playback_obj, casting it to the appropriate playback object subclass. For these casts to make sense, the two class hierarchies need to have the same structure.Note that the playback objects that
m_playback_objpoints to are GC-allocated, but the recording objects don’t own references: these associations only exist within a part of the code where the GC doesn’t collect, and are set back to NULL before the GC can run.
Another way to understand the structure of the code is to enable logging, via gcc_jit_context_set_logfile(). Here is an example of a log generated via this call:
JIT: libgccjit (GCC) version 6.0.0 20150803 (experimental) (x86_64-pc-linux-gnu) JIT: compiled by GNU C version 4.8.3 20140911 (Red Hat 4.8.3-7), GMP version 5.1.2, MPFR version 3.1.2, MPC version 1.0.1 JIT: entering: gcc_jit_context_set_str_option JIT: GCC_JIT_STR_OPTION_PROGNAME: "./test-hello-world.c.exe" JIT: exiting: gcc_jit_context_set_str_option JIT: entering: gcc_jit_context_set_int_option JIT: GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL: 3 JIT: exiting: gcc_jit_context_set_int_option JIT: entering: gcc_jit_context_set_bool_option JIT: GCC_JIT_BOOL_OPTION_DEBUGINFO: true JIT: exiting: gcc_jit_context_set_bool_option JIT: entering: gcc_jit_context_set_bool_option JIT: GCC_JIT_BOOL_OPTION_DUMP_INITIAL_TREE: false JIT: exiting: gcc_jit_context_set_bool_option JIT: entering: gcc_jit_context_set_bool_option JIT: GCC_JIT_BOOL_OPTION_DUMP_INITIAL_GIMPLE: false JIT: exiting: gcc_jit_context_set_bool_option JIT: entering: gcc_jit_context_set_bool_option JIT: GCC_JIT_BOOL_OPTION_SELFCHECK_GC: true JIT: exiting: gcc_jit_context_set_bool_option JIT: entering: gcc_jit_context_set_bool_option JIT: GCC_JIT_BOOL_OPTION_DUMP_SUMMARY: false JIT: exiting: gcc_jit_context_set_bool_option JIT: entering: gcc_jit_context_get_type JIT: exiting: gcc_jit_context_get_type JIT: entering: gcc_jit_context_get_type JIT: exiting: gcc_jit_context_get_type JIT: entering: gcc_jit_context_new_param JIT: exiting: gcc_jit_context_new_param JIT: entering: gcc_jit_context_new_function JIT: exiting: gcc_jit_context_new_function JIT: entering: gcc_jit_context_new_param JIT: exiting: gcc_jit_context_new_param JIT: entering: gcc_jit_context_get_type JIT: exiting: gcc_jit_context_get_type JIT: entering: gcc_jit_context_new_function JIT: exiting: gcc_jit_context_new_function JIT: entering: gcc_jit_context_new_string_literal JIT: exiting: gcc_jit_context_new_string_literal JIT: entering: gcc_jit_function_new_block JIT: exiting: gcc_jit_function_new_block JIT: entering: gcc_jit_block_add_comment JIT: exiting: gcc_jit_block_add_comment JIT: entering: gcc_jit_context_new_call JIT: exiting: gcc_jit_context_new_call JIT: entering: gcc_jit_block_add_eval JIT: exiting: gcc_jit_block_add_eval JIT: entering: gcc_jit_block_end_with_void_return JIT: exiting: gcc_jit_block_end_with_void_return JIT: entering: gcc_jit_context_dump_reproducer_to_file JIT: entering: void gcc::jit::recording::context::dump_reproducer_to_file(const char*) JIT: exiting: void gcc::jit::recording::context::dump_reproducer_to_file(const char*) JIT: exiting: gcc_jit_context_dump_reproducer_to_file JIT: entering: gcc_jit_context_compile JIT: in-memory compile of ctxt: 0x1283e20 JIT: entering: gcc::jit::result* gcc::jit::recording::context::compile() JIT: GCC_JIT_STR_OPTION_PROGNAME: "./test-hello-world.c.exe" JIT: GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL: 3 JIT: GCC_JIT_BOOL_OPTION_DEBUGINFO: true JIT: GCC_JIT_BOOL_OPTION_DUMP_INITIAL_TREE: false JIT: GCC_JIT_BOOL_OPTION_DUMP_INITIAL_GIMPLE: false JIT: GCC_JIT_BOOL_OPTION_DUMP_GENERATED_CODE: false JIT: GCC_JIT_BOOL_OPTION_DUMP_SUMMARY: false JIT: GCC_JIT_BOOL_OPTION_DUMP_EVERYTHING: false JIT: GCC_JIT_BOOL_OPTION_SELFCHECK_GC: true JIT: GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES: false JIT: gcc_jit_context_set_bool_allow_unreachable_blocks: false JIT: gcc_jit_context_set_bool_use_external_driver: false JIT: entering: void gcc::jit::recording::context::validate() JIT: exiting: void gcc::jit::recording::context::validate() JIT: entering: gcc::jit::playback::context::context(gcc::jit::recording::context*) JIT: exiting: gcc::jit::playback::context::context(gcc::jit::recording::context*) JIT: entering: gcc::jit::playback::compile_to_memory::compile_to_memory(gcc::jit::recording::context*) JIT: exiting: gcc::jit::playback::compile_to_memory::compile_to_memory(gcc::jit::recording::context*) JIT: entering: void gcc::jit::playback::context::compile() JIT: entering: gcc::jit::tempdir::tempdir(gcc::jit::logger*, int) JIT: exiting: gcc::jit::tempdir::tempdir(gcc::jit::logger*, int) JIT: entering: bool gcc::jit::tempdir::create() JIT: m_path_template: /tmp/libgccjit-XXXXXX JIT: m_path_tempdir: /tmp/libgccjit-CKq1M9 JIT: exiting: bool gcc::jit::tempdir::create() JIT: entering: void gcc::jit::playback::context::acquire_mutex() JIT: exiting: void gcc::jit::playback::context::acquire_mutex() JIT: entering: void gcc::jit::playback::context::make_fake_args(vec<char*>*, const char*, vec<gcc::jit::recording::requested_dump>*) JIT: reusing cached configure-time options JIT: configure_time_options[0]: -mtune=generic JIT: configure_time_options[1]: -march=x86-64 JIT: exiting: void gcc::jit::playback::context::make_fake_args(vec<char*>*, const char*, vec<gcc::jit::recording::requested_dump>*) JIT: entering: toplev::main JIT: argv[0]: ./test-hello-world.c.exe JIT: argv[1]: /tmp/libgccjit-CKq1M9/fake.c JIT: argv[2]: -fPIC JIT: argv[3]: -O3 JIT: argv[4]: -g JIT: argv[5]: -quiet JIT: argv[6]: --param JIT: argv[7]: ggc-min-expand=0 JIT: argv[8]: --param JIT: argv[9]: ggc-min-heapsize=0 JIT: argv[10]: -mtune=generic JIT: argv[11]: -march=x86-64 JIT: entering: bool jit_langhook_init() JIT: exiting: bool jit_langhook_init() JIT: entering: void gcc::jit::playback::context::replay() JIT: entering: void gcc::jit::recording::context::replay_into(gcc::jit::replayer*) JIT: exiting: void gcc::jit::recording::context::replay_into(gcc::jit::replayer*) JIT: entering: void gcc::jit::recording::context::disassociate_from_playback() JIT: exiting: void gcc::jit::recording::context::disassociate_from_playback() JIT: entering: void gcc::jit::playback::context::handle_locations() JIT: exiting: void gcc::jit::playback::context::handle_locations() JIT: entering: void gcc::jit::playback::function::build_stmt_list() JIT: exiting: void gcc::jit::playback::function::build_stmt_list() JIT: entering: void gcc::jit::playback::function::build_stmt_list() JIT: exiting: void gcc::jit::playback::function::build_stmt_list() JIT: entering: void gcc::jit::playback::function::postprocess() JIT: exiting: void gcc::jit::playback::function::postprocess() JIT: entering: void gcc::jit::playback::function::postprocess() JIT: exiting: void gcc::jit::playback::function::postprocess() JIT: exiting: void gcc::jit::playback::context::replay() JIT: exiting: toplev::main JIT: entering: void gcc::jit::playback::context::extract_any_requested_dumps(vec<gcc::jit::recording::requested_dump>*) JIT: exiting: void gcc::jit::playback::context::extract_any_requested_dumps(vec<gcc::jit::recording::requested_dump>*) JIT: entering: toplev::finalize JIT: exiting: toplev::finalize JIT: entering: virtual void gcc::jit::playback::compile_to_memory::postprocess(const char*) JIT: entering: void gcc::jit::playback::context::convert_to_dso(const char*) JIT: entering: void gcc::jit::playback::context::invoke_driver(const char*, const char*, const char*, timevar_id_t, bool, bool) JIT: entering: void gcc::jit::playback::context::add_multilib_driver_arguments(vec<char*>*) JIT: exiting: void gcc::jit::playback::context::add_multilib_driver_arguments(vec<char*>*) JIT: argv[0]: x86_64-unknown-linux-gnu-gcc-6.0.0 JIT: argv[1]: -m64 JIT: argv[2]: -shared JIT: argv[3]: /tmp/libgccjit-CKq1M9/fake.s JIT: argv[4]: -o JIT: argv[5]: /tmp/libgccjit-CKq1M9/fake.so JIT: argv[6]: -fno-use-linker-plugin JIT: entering: void gcc::jit::playback::context::invoke_embedded_driver(const vec<char*>*) JIT: exiting: void gcc::jit::playback::context::invoke_embedded_driver(const vec<char*>*) JIT: exiting: void gcc::jit::playback::context::invoke_driver(const char*, const char*, const char*, timevar_id_t, bool, bool) JIT: exiting: void gcc::jit::playback::context::convert_to_dso(const char*) JIT: entering: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() JIT: GCC_JIT_BOOL_OPTION_DEBUGINFO was set: handing over tempdir to jit::result JIT: entering: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) JIT: exiting: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) JIT: exiting: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() JIT: exiting: virtual void gcc::jit::playback::compile_to_memory::postprocess(const char*) JIT: entering: void gcc::jit::playback::context::release_mutex() JIT: exiting: void gcc::jit::playback::context::release_mutex() JIT: exiting: void gcc::jit::playback::context::compile() JIT: entering: gcc::jit::playback::context::~context() JIT: exiting: gcc::jit::playback::context::~context() JIT: exiting: gcc::jit::result* gcc::jit::recording::context::compile() JIT: gcc_jit_context_compile: returning (gcc_jit_result *)0x12f75d0 JIT: exiting: gcc_jit_context_compile JIT: entering: gcc_jit_result_get_code JIT: locating fnname: hello_world JIT: entering: void* gcc::jit::result::get_code(const char*) JIT: exiting: void* gcc::jit::result::get_code(const char*) JIT: gcc_jit_result_get_code: returning (void *)0x7ff6b8cd87f0 JIT: exiting: gcc_jit_result_get_code JIT: entering: gcc_jit_context_release JIT: deleting ctxt: 0x1283e20 JIT: entering: gcc::jit::recording::context::~context() JIT: exiting: gcc::jit::recording::context::~context() JIT: exiting: gcc_jit_context_release JIT: entering: gcc_jit_result_release JIT: deleting result: 0x12f75d0 JIT: entering: virtual gcc::jit::result::~result() JIT: entering: gcc::jit::tempdir::~tempdir() JIT: unlinking .s file: /tmp/libgccjit-CKq1M9/fake.s JIT: unlinking .so file: /tmp/libgccjit-CKq1M9/fake.so JIT: removing tempdir: /tmp/libgccjit-CKq1M9 JIT: exiting: gcc::jit::tempdir::~tempdir() JIT: exiting: virtual gcc::jit::result::~result() JIT: exiting: gcc_jit_result_release JIT: gcc::jit::logger::~logger()