[PATCH 0/5] tools/bootconfig: Fix tracing script filters and histogram dependencies

[PATCH 0/5] tools/bootconfig: Fix tracing script filters and histogram dependencies

[Eduardo Mucelli Rezende Oliveira]

Hi Masami,

Thanks for the reviews and for picking up the previous fixes. Here are
five more script fixes, based on bootconfig/core as you suggested.

They address:
- The documented ftrace.notraces setting being ignored by bconf2ftrace.sh.
- Single quotes in event actions producing invalid exported bootconfig.
- Literal filter text being mistaken for histogram variable syntax.
- Valid histogram dependency chains exceeding the exporter's retry limit.
- Helper key lookups matching unrelated bootconfig keys.

The action-quoting patch follows up on the trigger-filter fix you picked
up. It uses the same warning approach for unrepresentable values that
you suggested for event filters. The histogram scan uses POSIX sed.

I tested the conversion and export fixes in temporary tracefs instances
on my Ubuntu machine, running kernel 7.0.0-38-generic, with recording
disabled. The tests cover each relevant step and the combined series.
The helper fix has focused checks, and all 81 combined fixture checks
also pass with POSIX sed.

Feedback is welcome.

Thanks,
Eduardo

Eduardo Mucelli Rezende Oliveira (5):
  tools/bootconfig: Read the documented ftrace.notraces key
  tools/bootconfig: Preserve quotes in exported event actions
  tools/bootconfig: Ignore filter strings when resolving histogram
    variables
  tools/bootconfig: Resolve histogram dependencies until no progress is
    made
  tools/bootconfig: Match literal keys in tracing script helpers

 tools/bootconfig/scripts/bconf2ftrace.sh |  2 +-
 tools/bootconfig/scripts/ftrace2bconf.sh | 27 ++++++++++++++++++------
 tools/bootconfig/scripts/xbc.sh          | 13 ++++++++----
 3 files changed, 31 insertions(+), 11 deletions(-)


base-commit: a52ceed51e9bf8f1ca9c2c868b14bf6861986c65
-- 
2.53.0

[PATCH 1/5] tools/bootconfig: Read the documented ftrace.notraces key

[Eduardo Mucelli Rezende Oliveira]

The boot-time tracing documentation and trace_boot_set_ftrace_filter()
use ftrace.notraces for function exclusions. bconf2ftrace.sh instead looks
up ftrace.notrace, so the documented option produces no writes to
set_ftrace_notrace. This affects both the root instance and named ones.

Read the documented key so that the script applies the same exclusions
as the in-kernel bootconfig consumer.

Fixes: 7e66ef0046cc ("tools/bootconfig: Add a script to generate ftrace shell-command from bootconfig")
Link: https://docs.kernel.org/trace/boottime-trace.html#ftrace-per-instance-options
Signed-off-by: Eduardo Mucelli Rezende Oliveira <edumucelli@gmail.com>
---
I tested the documented ftrace.notraces setting with one and two function
names in a temporary tracefs instance on my Ubuntu machine, running
kernel 7.0.0-38-generic, with recording disabled. The original script
installed neither exclusion; the fixed script installed both exactly.
Fixture checks also cover the root instance, named instances, arrays,
an absent setting and the unchanged positive-filter path.

 tools/bootconfig/scripts/bconf2ftrace.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/bootconfig/scripts/bconf2ftrace.sh b/tools/bootconfig/scripts/bconf2ftrace.sh
index 7745c97e7abd..e5a833171b57 100755
--- a/tools/bootconfig/scripts/bconf2ftrace.sh
+++ b/tools/bootconfig/scripts/bconf2ftrace.sh
@@ -258,7 +258,7 @@ setup_instance() { # [instance]
 	set_value_of ${instance}.tracer ${instancedir}/current_tracer
 	set_array_of ${instance}.ftrace.filters \
 		${instancedir}/set_ftrace_filter
-	set_array_of ${instance}.ftrace.notrace \
+	set_array_of ${instance}.ftrace.notraces \
 		${instancedir}/set_ftrace_notrace
 
 	if xbc_has_key ${instance}.alloc_snapshot; then
-- 
2.53.0

[PATCH 2/5] tools/bootconfig: Preserve quotes in exported event actions

[Eduardo Mucelli Rezende Oliveira]

Reading complete trigger lines preserves filters that can contain single
quotes. Enclosing every action in single bootconfig quotes then produces
invalid output for a valid trigger such as traceoff if prev_comm == 'abc'.

Use double quotes for actions containing only single quotes. Actions with
both quote types can be emitted without enclosing quotes when they have
no bootconfig delimiters. Warn and skip unrepresentable actions containing
both quote types and delimiters, following the event-filter handling.

Fixes: 6a5740440c47 ("tools/bootconfig: Preserve event trigger filters in ftrace2bconf.sh")
Signed-off-by: Eduardo Mucelli Rezende Oliveira <edumucelli@gmail.com>
---
I reproduced this with valid traceoff triggers using single-quoted
filters in a temporary tracefs instance on my Ubuntu machine, running
kernel 7.0.0-38-generic, with recording disabled.

The original export failed bootconfig parsing. With the fix, supported
single-quoted actions parsed and their decoded commands could be
restored unchanged to tracefs. Unrepresentable mixed-quote actions
produced a warning and were skipped. Mixed quotes without delimiters
and ordinary double-quoted filters were checked as well.

 tools/bootconfig/scripts/ftrace2bconf.sh | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/tools/bootconfig/scripts/ftrace2bconf.sh b/tools/bootconfig/scripts/ftrace2bconf.sh
index 31702be0c68d..77ec17c94a17 100755
--- a/tools/bootconfig/scripts/ftrace2bconf.sh
+++ b/tools/bootconfig/scripts/ftrace2bconf.sh
@@ -126,7 +126,22 @@ per_event_options() { # event-dir
 		*:count=*)
 			action="${action_cmd%:count=*}:${action_cmd##*:count=}$action_rest";;
 		esac
-		emit_kv $PREFIX.event.$group.$event.actions += "'$action'"
+		case "$action" in
+		*\'*)
+			case "$action" in
+			*\"*)
+				case "$action" in
+				*[,\;\#\}]*)
+					cat 1>&2 << EOF
+# WARN: $PREFIX.event.$group.$event.actions cannot be quoted for bootconfig. Skip it.
+EOF
+					continue;;
+				esac;;
+			*) action="\"$action\"";;
+			esac;;
+		*) action="'$action'";;
+		esac
+		emit_kv $PREFIX.event.$group.$event.actions += "$action"
 	done
 
 	if [ $GROUP_ENABLED -eq 0 ] && event_is_enabled $evdir/enable; then
-- 
2.53.0

[PATCH 3/5] tools/bootconfig: Ignore filter strings when resolving histogram variables

[Eduardo Mucelli Rezende Oliveira]

The histogram variable resolver scans the entire trigger line, including
its filter. A literal string such as "$not_a_variable" is consequently
read as a variable reference and the histogram is deferred and omitted.
Conversely, a filter containing ":dep0=" looks like a definition and can
cause a consumer to be exported before its real provider.

Remove the filter portion before looking for variable definitions and
references. Leave the complete filter intact when exporting the action.

Fixes: 2b86062a34a8 ("tools/bootconfig: Add a script to generates bootconfig from ftrace")
Signed-off-by: Eduardo Mucelli Rezende Oliveira <edumucelli@gmail.com>
---
I tested literal dollar signs and assignment-looking strings in
histogram filters in a temporary tracefs instance on my Ubuntu machine,
running kernel 7.0.0-38-generic, with recording disabled.

The original exporter omitted the dollar-sign cases and treated text
inside another filter as a provider, putting a consumer first. With
the fix, the literal filters were retained and the real provider was
exported before its consumer.

 tools/bootconfig/scripts/ftrace2bconf.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/bootconfig/scripts/ftrace2bconf.sh b/tools/bootconfig/scripts/ftrace2bconf.sh
index 77ec17c94a17..8ba1a698611e 100755
--- a/tools/bootconfig/scripts/ftrace2bconf.sh
+++ b/tools/bootconfig/scripts/ftrace2bconf.sh
@@ -86,10 +86,10 @@ synth_event_options() {
 # Variables resolver
 
 defined_vars() { # event-dir
-	grep "^hist" $1/trigger | grep -o ':[a-zA-Z0-9_]*='
+	grep "^hist" $1/trigger | sed 's/ if .*//' | grep -o ':[a-zA-Z0-9_]*='
 }
 referred_vars() {
-	grep "^hist" $1/trigger | grep -o '$[a-zA-Z0-9_]*'
+	grep "^hist" $1/trigger | sed 's/ if .*//' | grep -o '$[a-zA-Z0-9_]*'
 }
 
 event_is_enabled() { # enable-file
-- 
2.53.0

[PATCH 4/5] tools/bootconfig: Resolve histogram dependencies until no progress is made

[Eduardo Mucelli Rezende Oliveira]

The exporter retries unresolved events only three times. A valid chain
of eight histogram dependencies in reverse traversal order therefore
exports only four histograms and reports the others as undefined, even
though all their providers are present.

Retry while the unresolved event list changes. Stop when all events have
been exported or a pass makes no progress, preserving the diagnostic for
missing variables and dependency cycles.

Fixes: 2b86062a34a8 ("tools/bootconfig: Add a script to generates bootconfig from ftrace")
Signed-off-by: Eduardo Mucelli Rezende Oliveira <edumucelli@gmail.com>
---
I installed a valid chain of eight dependent histograms in a temporary
tracefs instance on my Ubuntu machine, running kernel 7.0.0-38-generic,
with recording disabled.

The original exporter emitted only four histograms, so replay restored
half the configuration. With the fix, all eight were exported in provider
order and restored correctly. Fixture checks cover shorter and longer
chains, missing variables and cycles that must stop without looping.

 tools/bootconfig/scripts/ftrace2bconf.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/bootconfig/scripts/ftrace2bconf.sh b/tools/bootconfig/scripts/ftrace2bconf.sh
index 8ba1a698611e..b3c976bc98c8 100755
--- a/tools/bootconfig/scripts/ftrace2bconf.sh
+++ b/tools/bootconfig/scripts/ftrace2bconf.sh
@@ -203,10 +203,10 @@ event_options() {
 			per_event_options $INSTANCE/events/$group/$event
 		done
 	done
-	retry=0
-	while [ $retry -lt 3 ]; do
+	while [ "$UNRESOLVED_EVENTS" ]; do
+		prev_unresolved=$UNRESOLVED_EVENTS
 		retry_unresolved
-		retry=$((retry + 1))
+		[ "$prev_unresolved" = "$UNRESOLVED_EVENTS" ] && break
 	done
 	if [ "$UNRESOLVED_EVENTS" ]; then
 		cat 1>&2 << EOF
-- 
2.53.0

[PATCH 5/5] tools/bootconfig: Match literal keys in tracing script helpers

[Eduardo Mucelli Rezende Oliveira]

The xbc shell helpers interpolate bootconfig keys into regular expressions.
Dots in keys consequently match arbitrary characters: asking for
ftrace.options can also return ftraceXoptions. Branch and subkey lookups
also match unrelated keys that merely share the requested prefix.

Compare complete keys literally and require a dot at descendant boundaries.
Keep the optional subkey pattern operating on the suffix only.

Fixes: 7e66ef0046cc ("tools/bootconfig: Add a script to generate ftrace shell-command from bootconfig")
Signed-off-by: Eduardo Mucelli Rezende Oliveira <edumucelli@gmail.com>
---
The helper checks cover literal dotted keys, missing keys, branch
boundaries and subkey selection. The original helpers passed one of
five checks; the fixed helpers pass all five. These checks also pass
when combined with the other fixes in this series.

 tools/bootconfig/scripts/xbc.sh | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/bootconfig/scripts/xbc.sh b/tools/bootconfig/scripts/xbc.sh
index 1f0ebf50dd2d..16ee590661b4 100644
--- a/tools/bootconfig/scripts/xbc.sh
+++ b/tools/bootconfig/scripts/xbc.sh
@@ -36,21 +36,26 @@ xbc_get_val() { # key [maxnum]
 	if [ "$2" ]; then
 		MAXOPT="-L $2"
 	fi
-	grep "^$1 =" $XBC_TMPFILE | cut -d= -f2- | \
+	awk -v key="$1" '$1 == key && $2 == "="' "$XBC_TMPFILE" | cut -d= -f2- | \
 		sed -e 's/", /" /g' -e "s/',/' /g" | \
 		xargs $MAXOPT -n 1 echo
 }
 
 xbc_has_key() { # key
-	grep -q "^$1 =" $XBC_TMPFILE
+	awk -v key="$1" '$1 == key && $2 == "=" { found = 1 }
+		END { exit !found }' "$XBC_TMPFILE"
 }
 
 xbc_has_branch() { # prefix-key
-	grep -q "^$1" $XBC_TMPFILE
+	awk -v key="$1" '$1 == key || index($1, key ".") == 1 { found = 1 }
+		END { exit !found }' "$XBC_TMPFILE"
 }
 
 xbc_subkeys() { # prefix-key depth [subkey-pattern]
 	__keys=`echo $1 | sed "s/\./ /g"`
 	__s=`nr_args $__keys`
-	grep "^$1$3" $XBC_TMPFILE | cut -d= -f1| cut -d. -f$((__s + 1))-$((__s + $2)) | uniq
+	awk -v key="$1" -v pattern="$3" '
+		index($1, key ".") == 1 &&
+		substr($1, length(key) + 1) ~ ("^" pattern) { print $1 }
+	' "$XBC_TMPFILE" | cut -d. -f$((__s + 1))-$((__s + $2)) | uniq
 }
-- 
2.53.0