Custom keys

Chalk allows you to configure reports (what key data we collect), sinks (where reports are sent), and custom keys (what entirely new data can we enable reports to collect).

This guide is about custom keys (what entirely new data can we enable reports to collect).

Our first custom key (with a hardcoded value)

First let’s add a custom key with a hardcoded value. Whenever chalk inserts or reports this metadata key, we’ll see the fixed value.

$ cat test1.c4m
# Hardcode a value.
keyspec X_VALUE {
  kind:     ChalkTimeHost
  type:     string
  value:    "hello"
}

# Enable the key in the report template used by `chalk insert`.
report_template.insertion_default.key.X_VALUE.use: true

As you’ll see in Reports, defining the custom key is not enough, it must be included in a report. And that is what we’ve done in the last line of the above file.

Now let’s reset Chalk’s config, clear out old logs, load this new config, and Chalk mark a copy of the cp binary.

chalk load default --replace
chalk load test1.c4m
cp $(which cp) cp2
chalk insert cp2

And you can find the key included in reports in ~/.local/chalk/chalk.log.

The report is now in ~/.local/chalk/chalk.log and contains X_VALUE:

$ tail -1 ~/.local/chalk/chalk.log | jq '.[0].X_VALUE'
"hello"

Dynamic keys

With Chalk, you can do more dynamic collection as well. For instance, you can collect the value of a specific environment variable:

$ cat test2.c4m
# Get a value from an environment variable `ENV_VAR`.
keyspec X_ENV_VAR {
  kind:     ChalkTimeHost
  type:     string
  value:    env("ENV_VAR")
}

# Enable the new key in the insert report template.
report_template.insertion_default.key.X_ENV_VAR.use: true

You can see the use of a builtin function named env. Chalk has many such functions. You can see full documentation from the command line by typing chalk help builtins; they will be presented in tables, sorted by category.

Reset Chalk to a clean slate, then load and run with ENV_VAR set:

chalk load default --replace
chalk load test2.c4m
ENV_VAR=world chalk insert cp2

~/.local/chalk/chalk.log now contains the value pulled from the environment:

$ tail -1 ~/.local/chalk/chalk.log | jq '.[0].X_ENV_VAR'
"world"

Shelling out and other function calls

The run builtin allows us to easily capture all stdout and stderr for a shell command. We can combine that with the strip builtin, which removes leading and trailing white space:

$ cat test3.c4m
# Get a value from the output of a system command.
keyspec X_CMD {
  kind:     ChalkTimeHost
  type:     string
  value:    strip(run("echo hello!"))
}

# Enable the new key in the insert report template.
report_template.insertion_default.key.X_CMD.use: true

Reset Chalk to a clean slate, then load and run:

chalk load default --replace
chalk load test3.c4m
chalk insert cp2

X_CMD now appears in the report:

$ tail -1 ~/.local/chalk/chalk.log | jq '.[0].X_CMD'
"hello!"

Callbacks

Using the value field has a few limitations:

• Everything you need to do must be doable as a single expression; you can’t add a block of code here.
• The code you add will get evaluated on every single run of Chalk, even if your program doesn’t end up asking for the X_CMD key. In some sense, that should be expected if you think of your config file as a program, where we are assigning to the keyspec.X_CMD.value attribute.

Both of these issues can be addressed if we create a simple function. Let’s define a new keyspec object that makes use of one:

$ cat test4.c4m
# For more advanced keys, if you need to do any special processing
# you can define a value callback where you can have more advanced logic.
keyspec X_FUNC {
  kind:     ChalkTimeHost
  type:     string
  callback: func key_callback
}

func key_callback(contexts) {
  return "hello " + env("ENV_VAR")
}

# Enable the new key in the insert report template.
report_template.insertion_default.key.X_FUNC.use: true

Instead of using the value property, we set the callback property. These two properties are mutually exclusive. Your configuration will not validate if you do both.

The contexts parameter is a colon-separated list of any contexts, which, if using Chalk with docker, will be the available docker contexts. This is a string, and your callback must accept one and only one string. The return value in this case is also a string because we defined our keyspec to take a string. All of these types are automatically validated when you try to load your configuration. They do not need to be specified, but you may do so if you like:

func key_callback(contexts : string) -> string {
  return "hello " + env("ENV_VAR")
}

Reset Chalk to a clean slate, then load and run:

chalk load default --replace
chalk load test4.c4m
ENV_VAR=world chalk insert cp2

X_FUNC now shows the callback’s return value in ~/.local/chalk/chalk.log:

$ tail -1 ~/.local/chalk/chalk.log | jq '.[0].X_FUNC'
"hello world"

Per-artifact keys

So far in this document, every key has used kind: ChalkTimeHost, which collects a single value for the whole run. This kind of key appears at the top level of the report (alongside _OPERATION, _CHALKS, etc.). If you’d rather have the key reported once per artifact (which shows up within the _CHALKS key in the JSON) use kind: ChalkTimeArtifact instead.

A situation where you might want this is if you’re Chalk marking multi-arch Docker images. ChalkTimeArtifact keys will get evaluated separately for each architecture, while ChalkTimeHost will get evaluated once for all architectures.

$ cat test5.c4m
# Per-artifact key: reported once for every chalked artifact.
keyspec X_ARTIFACT {
  kind:     ChalkTimeArtifact
  type:     string
  value:    "per-artifact-hello"
}

# Enable the key in the insert report template.
report_template.insertion_default.key.X_ARTIFACT.use: true

Reset Chalk to a clean slate, then load and run:

chalk load default --replace
chalk load test5.c4m
chalk insert cp2

X_ARTIFACT now lives inside the _CHALKS array entry for cp2:

$ tail -1 ~/.local/chalk/chalk.log | jq '.[0]._CHALKS[0].X_ARTIFACT'
"per-artifact-hello"

Reporting custom keys in other contexts

You can also enable your custom keys for docker labels, docker chalk marks, and the terminal output. You can combine all of these in one configuration.

Add your custom keys to Docker labels

Chalk has a built in reporting configuration called chalk_labels that we can add our keys to. The below config augments what is already added to the chalk_labels template, it doesn’t replace it:

# Automatically add these keys as docker labels during docker build wrapping.
# By default, docker uses the "chalk_labels" mark template for that.
# See:
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/chalk.c42spec#L1659-L1675
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/base_chalk_templates.c4m#L421-L437
mark_template chalk_labels {
  key.X_FUNC.use    = true
}

Add your custom keys to the Docker chalk mark

Chalk also uses reporting templates to figure out what to stick in a chalk mark. By default, the predefined template used when chalking docker containers is called mark_default and you can add to it like so:

# Automatically add these keys to the chalk mark embedded in the docker image.
# By default, docker wrapping uses the "mark_default" template for embedded chalkmarks.
# See:
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/base_outconf.c4m#L20-L23
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/base_chalk_templates.c4m#L244-L330
mark_template mark_default {
  key.X_FUNC.use    = true
}

Showing your keys on the command line

You might want your keys to show up on the summary report we print on the command line when you are manually playing around with the command. The summary report changes based on the command, but the defaults are that operations that insert chalk marks use the terminal_insert template, and all other operations use terminal_rest.

By the way, if you want other keys to show in this report that don’t already show, you can add them to the list. Or, turn them off if they’re on, by setting the value to false.

Finally, remember that, since we’re adding chalk-time keys in these examples, even with the below configuration, they would only show up when running chalk extract if they are added to the chalk mark.

If you want different behavior, also create a custom run-time key.

# Show these values in terminal output when running docker build.
# The terminal output is reported via a custom report which uses the "terminal_insert" reporting template,
# and "json_console_out" which sends output to stdout.
# See:
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/ioconfig.c4m#L11-L16
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/base_report_templates.c4m#L2015-L2113
report_template terminal_insert {
  key.X_FUNC.use    = true
}

# Show these values in terminal output when running `chalk extract` on the built image.
# Similar to the above, the non-insert operations have a different custom report,
# which uses the "terminal_rest" reporting template.
# See:
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/ioconfig.c4m#L18-L23
# https://github.com/crashappsec/chalk/blob/f8124016855e5a10ab7995f3a8bdfc9e08f06042/src/configs/base_report_templates.c4m#L2115-L2219
report_template terminal_rest {
  key.X_FUNC.use    = true
}

Using custom keys with Docker

For docker workflows, first turn on docker entry point wrapping. Run the following command to import the module that turns on docker entry point wrapping:

chalk load https://chalkdust.io/wrap_entrypoints.c4m

When you run this command, you should see something like:

 Configuring Component: https://chalkdust.io/wrap_entrypoints
 Finished configuration for https://chalkdust.io/wrap_entrypoints
info:  [testing config]: Validating configuration.
info:  [testing config]: Configuration successfully validated.
info:  Configuration replaced in binary: /root/Code/co/chalk/chalk
info:  /root/.local/chalk/chalk.log: Open (sink conf='default_out')
info:  Full chalk report appended to: ~/.local/chalk/chalk.log

This only needs to be done once; chalk will add the module to its own internal chalk mark, and automatically load it in future runs.

Now we can do a docker build which will wrap the image and insert /chalk.json:

$ echo FROM alpine | ENV_VAR=world ./chalk docker build -f - . -t chalk_custom_keys
...
[
  {
    "_OPERATION": "build",
    "_CHALKS": [
      {
        "CHALK_ID": "XHY0JF-KCER-KJPH-R3DHCZ",
        "METADATA_ID": "39C51D-076C-PNDM-SS8WAA",
        ...
      }
    ],
    "X_FUNC": "hello world",
    ...
  }
]

Inspect image

Now that we have the chalk_custom_keys image, let’s extract the chalk mark out of it:

$ chalk --log-level=none extract chalk_custom_keys
[
  {
    "_OPERATION": "extract",
    "_CHALKS": [
      {
        "CHALK_ID": "XHY0JF-KCER-KJPH-R3DHCZ",
        "X_FUNC": "hello world",
        ...
      }
    ],
    ...
  }
]

You can see that the chalk mark contains the same custom keys.

We can also directly inspect the /chalk.json too:

$ docker run -it --rm --entrypoint=cat chalk_custom_keys /chalk.json | jq
{
  "X_FUNC": "hello world",
  ...
}

Finally let’s see the labels for the image:

$ docker image inspect chalk_custom_keys | jq '.[].Config.Labels'
{
  ...
  "run.crashoverride.x-cmd": "world",
  "run.crashoverride.x-env-var": "world",
  "run.crashoverride.x-func": "hello world",
  "run.crashoverride.x-value": "hello"
}

Properties of custom keys

A custom key is defined by a keyspec block. The fields available are:

The value or callback fields must be present (mutually exclusive).

kind

kind is an enumeration with four values that control when data is collected and what scope the value belongs to:

Currently, Chalk requires that any custom key begins either with X_ or _X_. Chalk-time keys (X_…) can be embedded in the Chalk mark itself so they’re available later from chalk extract. Run-time keys (_X_…) are never embedded. They are only ever present in reports for the operation that produced them. That is why the prefixes differ: the leading underscore signals “unchalkable”.

The use of the letter X is meant to evoke the word extension, and is required to avoid accidental collisions as we expand the kinds of metadata that Chalk can collect. We chose this because it is common, in that MIME essentially does the same thing.

type

We recommend you keep everything strings, meaning you will set the type field to string. Your strings will get encoded as JSON when reporting.