* [PATCH v2 1/3] drm/ci: uprev mesa
2025-02-05 8:16 [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Vignesh Raman
@ 2025-02-05 8:16 ` Vignesh Raman
2025-02-05 8:16 ` [PATCH v2 2/3] drm/ci: update gitlab rules Vignesh Raman
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Vignesh Raman @ 2025-02-05 8:16 UTC (permalink / raw)
To: dri-devel
Cc: daniels, helen.koike, airlied, simona.vetter, robdclark,
guilherme.gallo, sergi.blanch.torne, valentine.burley,
jani.nikula, dmitry.baryshkov, mripard, linux-kernel
Uprev mesa to adapt to the latest changes in mesa ci
which includes new container jobs and stages. Also update
lava-submit script to adapt to the recent changes in mesa
to use LAVA rootfs overlays.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
---
v2:
- Uprev mesa to the latest version and update the yaml files and scripts.
---
drivers/gpu/drm/ci/build.sh | 2 +-
drivers/gpu/drm/ci/build.yml | 103 +++++++++++++++++++++++++++++-
drivers/gpu/drm/ci/container.yml | 22 +++----
drivers/gpu/drm/ci/gitlab-ci.yml | 81 +++++++++++++++--------
drivers/gpu/drm/ci/igt_runner.sh | 13 ++--
drivers/gpu/drm/ci/image-tags.yml | 11 +++-
drivers/gpu/drm/ci/lava-submit.sh | 100 +++++++++++++++++++++--------
drivers/gpu/drm/ci/test.yml | 17 +++--
8 files changed, 264 insertions(+), 85 deletions(-)
diff --git a/drivers/gpu/drm/ci/build.sh b/drivers/gpu/drm/ci/build.sh
index 139b81db6312..19fe01257ab9 100644
--- a/drivers/gpu/drm/ci/build.sh
+++ b/drivers/gpu/drm/ci/build.sh
@@ -132,7 +132,7 @@ fi
# Pass needed files to the test stage
mkdir -p install
cp -rfv .gitlab-ci/* install/.
-cp -rfv ci/* install/.
+cp -rfv bin/ci/* install/.
cp -rfv install/common install/ci-common
cp -rfv drivers/gpu/drm/ci/* install/.
diff --git a/drivers/gpu/drm/ci/build.yml b/drivers/gpu/drm/ci/build.yml
index 9c198239033d..a2baa00a20db 100644
--- a/drivers/gpu/drm/ci/build.yml
+++ b/drivers/gpu/drm/ci/build.yml
@@ -2,7 +2,7 @@
extends:
- .build-rules
- .container+build-rules
- stage: build
+ stage: build-only
artifacts:
paths:
- artifacts
@@ -110,3 +110,104 @@ build-nodebugfs:arm64:
build:x86_64:
extends: .build:x86_64
+
+# Disable build jobs that we won't use
+alpine-build-testing:
+ rules:
+ - when: never
+
+debian-android:
+ rules:
+ - when: never
+
+debian-arm32:
+ rules:
+ - when: never
+
+debian-arm32-asan:
+ rules:
+ - when: never
+
+debian-arm64:
+ rules:
+ - when: never
+
+debian-arm64-asan:
+ rules:
+ - when: never
+
+debian-arm64-build-test:
+ rules:
+ - when: never
+
+debian-arm64-release:
+ rules:
+ - when: never
+
+debian-build-testing:
+ rules:
+ - when: never
+
+debian-clang:
+ rules:
+ - when: never
+
+debian-clang-release:
+ rules:
+ - when: never
+
+debian-no-libdrm:
+ rules:
+ - when: never
+
+debian-ppc64el:
+ rules:
+ - when: never
+
+debian-release:
+ rules:
+ - when: never
+
+debian-s390x:
+ rules:
+ - when: never
+
+debian-testing:
+ rules:
+ - when: never
+
+debian-testing-asan:
+ rules:
+ - when: never
+
+debian-testing-msan:
+ rules:
+ - when: never
+
+debian-vulkan:
+ rules:
+ - when: never
+
+debian-x86_32:
+ rules:
+ - when: never
+
+fedora-release:
+ rules:
+ - when: never
+
+rustfmt:
+ rules:
+ - when: never
+
+shader-db:
+ rules:
+ - when: never
+
+windows-msvc:
+ rules:
+ - when: never
+
+yaml-toml-shell-py-test:
+ rules:
+ - when: never
diff --git a/drivers/gpu/drm/ci/container.yml b/drivers/gpu/drm/ci/container.yml
index 2a94f54ce4cf..07dc13ff865d 100644
--- a/drivers/gpu/drm/ci/container.yml
+++ b/drivers/gpu/drm/ci/container.yml
@@ -24,7 +24,7 @@ alpine/x86_64_build:
rules:
- when: never
-debian/x86_64_test-vk:
+debian/arm64_test-gl:
rules:
- when: never
@@ -32,7 +32,15 @@ debian/arm64_test-vk:
rules:
- when: never
-debian/arm64_test-gl:
+debian/ppc64el_build:
+ rules:
+ - when: never
+
+debian/s390x_build:
+ rules:
+ - when: never
+
+debian/x86_64_test-vk:
rules:
- when: never
@@ -56,14 +64,6 @@ windows_test_msvc:
rules:
- when: never
-.debian/x86_64_build-mingw:
- rules:
- - when: never
-
-rustfmt:
- rules:
- - when: never
-
windows_msvc:
rules:
- - when: never
\ No newline at end of file
+ - when: never
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
index b7214718723c..48ab2de9c90b 100644
--- a/drivers/gpu/drm/ci/gitlab-ci.yml
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -1,6 +1,6 @@
variables:
DRM_CI_PROJECT_PATH: &drm-ci-project-path mesa/mesa
- DRM_CI_COMMIT_SHA: &drm-ci-commit-sha c6a9a9c3bce90923f7700219354e0b6e5a3c9ba6
+ DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 7d3062470f3ccc6cb40540e772e902c7e2248024
UPSTREAM_REPO: https://gitlab.freedesktop.org/drm/kernel.git
TARGET_BRANCH: drm-next
@@ -20,6 +20,9 @@ variables:
rm download-git-cache.sh
set +o xtrace
S3_JWT_FILE: /s3_jwt
+ S3_JWT_FILE_SCRIPT: |-
+ echo -n '${S3_JWT}' > '${S3_JWT_FILE}' &&
+ unset CI_JOB_JWT S3_JWT # Unsetting vulnerable env variables
S3_HOST: s3.freedesktop.org
# This bucket is used to fetch the kernel image
S3_KERNEL_BUCKET: mesa-rootfs
@@ -31,11 +34,7 @@ variables:
PIPELINE_ARTIFACTS_BASE: ${S3_HOST}/${S3_ARTIFACTS_BUCKET}/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
# per-job artifact storage on MinIO
JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
- # default kernel for rootfs before injecting the current kernel tree
- KERNEL_REPO: "gfx-ci/linux"
- KERNEL_TAG: "v6.6.21-mesa-f8ea"
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG}
- PKG_REPO_REV: "3cc12a2a"
LAVA_TAGS: subset-1-gfx
LAVA_JOB_PRIORITY: 30
ARTIFACTS_BASE_URL: https://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts
@@ -50,16 +49,13 @@ default:
- export SCRIPTS_DIR=$(mktemp -d)
- curl -L -s --retry 4 -f --retry-all-errors --retry-delay 60 -O --output-dir "${SCRIPTS_DIR}" "${DRM_CI_PROJECT_URL}/-/raw/${DRM_CI_COMMIT_SHA}/.gitlab-ci/setup-test-env.sh"
- source ${SCRIPTS_DIR}/setup-test-env.sh
- - echo -e "\e[0Ksection_start:$(date +%s):unset_env_vars_section[collapsed=true]\r\e[0KUnsetting vulnerable environment variables"
- - echo -n "${S3_JWT}" > "${S3_JWT_FILE}"
- - unset CI_JOB_JWT S3_JWT
- - echo -e "\e[0Ksection_end:$(date +%s):unset_env_vars_section\r\e[0K"
+ - eval "$S3_JWT_FILE_SCRIPT"
- echo -e "\e[0Ksection_start:$(date +%s):drm_ci_download_section[collapsed=true]\r\e[0KDownloading mesa from $DRM_CI_PROJECT_URL/-/archive/$DRM_CI_COMMIT_SHA/mesa-$DRM_CI_COMMIT_SHA.tar.gz"
- cd $CI_PROJECT_DIR
- curl --output - $DRM_CI_PROJECT_URL/-/archive/$DRM_CI_COMMIT_SHA/mesa-$DRM_CI_COMMIT_SHA.tar.gz | tar -xz
- mv mesa-$DRM_CI_COMMIT_SHA/.gitlab-ci* .
- - mv mesa-$DRM_CI_COMMIT_SHA/bin/ci .
+ - mv mesa-$DRM_CI_COMMIT_SHA/bin .
- rm -rf mesa-$DRM_CI_COMMIT_SHA/
- echo -e "\e[0Ksection_end:$(date +%s):drm_ci_download_section\r\e[0K"
@@ -85,6 +81,7 @@ include:
- project: *drm-ci-project-path
ref: *drm-ci-commit-sha
file:
+ - '/.gitlab-ci/build/gitlab-ci.yml'
- '/.gitlab-ci/container/gitlab-ci.yml'
- '/.gitlab-ci/farm-rules.yml'
- '/.gitlab-ci/lava/lava-gitlab-ci.yml'
@@ -115,9 +112,10 @@ include:
stages:
- sanity
- container
- - code-validation
- git-archive
- - build
+ - build-for-tests
+ - build-only
+ - code-validation
- amdgpu
- i915
- mediatek
@@ -197,7 +195,7 @@ stages:
.ci-deqp-artifacts:
artifacts:
- name: "mesa_${CI_JOB_NAME}"
+ name: "${CI_PROJECT_NAME}_${CI_JOB_NAME}"
when: always
untracked: false
paths:
@@ -264,30 +262,63 @@ sanity:
rules:
- if: *is-pre-merge
when: on_success
- # Other cases default to never
+ - when: never
variables:
GIT_STRATEGY: none
script:
# ci-fairy check-commits --junit-xml=check-commits.xml
- ci-fairy check-merge-request --require-allow-collaboration --junit-xml=check-merge-request.xml
+ - |
+ set -eu
+ image_tags=(
+ ALPINE_X86_64_LAVA_SSH_TAG
+ CONTAINER_TAG
+ DEBIAN_BASE_TAG
+ DEBIAN_BUILD_TAG
+ DEBIAN_PYUTILS_TAG
+ DEBIAN_TEST_GL_TAG
+ KERNEL_ROOTFS_TAG
+ KERNEL_TAG
+ PKG_REPO_REV
+ )
+ for var in "${image_tags[@]}"
+ do
+ if [ "$(echo -n "${!var}" | wc -c)" -gt 20 ]
+ then
+ echo "$var is too long; please make sure it is at most 20 chars."
+ exit 1
+ fi
+ done
artifacts:
when: on_failure
reports:
junit: check-*.xml
+ tags:
+ - placeholder-job
-# Rules for tests that should not block merging, but should be available to
-# optionally run with the "play" button in the UI in pre-merge non-marge
-# pipelines. This should appear in "extends:" after any includes of
-# test-source-dep.yml rules, so that these rules replace those.
-.test-manual-mr:
+
+mr-label-maker-test:
+ extends:
+ - .fdo.ci-fairy
+ stage: sanity
rules:
- - !reference [.no_scheduled_pipelines-rules, rules]
- - if: *is-forked-branch-or-pre-merge-not-for-marge
- when: manual
+ - !reference [.mr-label-maker-rules, rules]
variables:
- JOB_TIMEOUT: 80
-
+ GIT_STRATEGY: fetch
+ timeout: 10m
+ script:
+ - set -eu
+ - python3 -m venv .venv
+ - source .venv/bin/activate
+ - pip install git+https://gitlab.freedesktop.org/freedesktop/mr-label-maker
+ - mr-label-maker --dry-run --mr $CI_MERGE_REQUEST_IID
# Jobs that need to pass before spending hardware resources on further testing
.required-for-hardware-jobs:
- needs: []
+ needs:
+ - job: clang-format
+ optional: true
+ - job: rustfmt
+ optional: true
+ - job: toml-lint
+ optional: true
diff --git a/drivers/gpu/drm/ci/igt_runner.sh b/drivers/gpu/drm/ci/igt_runner.sh
index f38836ec837c..68b042e43b7f 100755
--- a/drivers/gpu/drm/ci/igt_runner.sh
+++ b/drivers/gpu/drm/ci/igt_runner.sh
@@ -47,7 +47,7 @@ else
ARCH="x86_64"
fi
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s ${FDO_HTTP_CACHE_URI:-}$PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd -v -x -C /
+curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s $PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd -v -x -C /
TESTLIST="/igt/libexec/igt-gpu-tools/ci-testlist.txt"
@@ -69,7 +69,7 @@ igt-runner \
run \
--igt-folder /igt/libexec/igt-gpu-tools \
--caselist $TESTLIST \
- --output /results \
+ --output $RESULTS_DIR \
-vvvv \
$IGT_SKIPS \
$IGT_FLAKES \
@@ -80,13 +80,10 @@ set -e
deqp-runner junit \
--testsuite IGT \
- --results /results/failures.csv \
- --output /results/junit.xml \
+ --results $RESULTS_DIR/failures.csv \
+ --output $RESULTS_DIR/junit.xml \
--limit 50 \
- --template "See https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/results/{{testcase}}.xml"
-
-# Store the results also in the simpler format used by the runner in ChromeOS CI
-#sed -r 's/(dmesg-warn|pass)/success/g' /results/results.txt > /results/results_simple.txt
+ --template "See $ARTIFACTS_BASE_URL/results/{{testcase}}.xml"
cd $oldpath
exit $ret
diff --git a/drivers/gpu/drm/ci/image-tags.yml b/drivers/gpu/drm/ci/image-tags.yml
index 8d8b9e71852e..20049f3626b2 100644
--- a/drivers/gpu/drm/ci/image-tags.yml
+++ b/drivers/gpu/drm/ci/image-tags.yml
@@ -1,5 +1,5 @@
variables:
- CONTAINER_TAG: "2024-09-09-uprevs"
+ CONTAINER_TAG: "20250204-mesa-uprev"
DEBIAN_X86_64_BUILD_BASE_IMAGE: "debian/x86_64_build-base"
DEBIAN_BASE_TAG: "${CONTAINER_TAG}"
@@ -7,9 +7,16 @@ variables:
DEBIAN_BUILD_TAG: "${CONTAINER_TAG}"
KERNEL_ROOTFS_TAG: "${CONTAINER_TAG}"
+ # default kernel for rootfs before injecting the current kernel tree
+ KERNEL_TAG: "v6.13-rc4-mesa-5e77"
+ KERNEL_REPO: "gfx-ci/linux"
+ PKG_REPO_REV: "bca9635d"
DEBIAN_X86_64_TEST_BASE_IMAGE: "debian/x86_64_test-base"
DEBIAN_X86_64_TEST_IMAGE_GL_PATH: "debian/x86_64_test-gl"
DEBIAN_TEST_GL_TAG: "${CONTAINER_TAG}"
- ALPINE_X86_64_LAVA_SSH_TAG: "${CONTAINER_TAG}"
\ No newline at end of file
+ DEBIAN_PYUTILS_IMAGE: "debian/x86_64_pyutils"
+ DEBIAN_PYUTILS_TAG: "${CONTAINER_TAG}"
+
+ ALPINE_X86_64_LAVA_SSH_TAG: "${CONTAINER_TAG}"
diff --git a/drivers/gpu/drm/ci/lava-submit.sh b/drivers/gpu/drm/ci/lava-submit.sh
index 6add15083c78..6e5ac51e8c0a 100755
--- a/drivers/gpu/drm/ci/lava-submit.sh
+++ b/drivers/gpu/drm/ci/lava-submit.sh
@@ -1,58 +1,102 @@
-#!/bin/bash
+#!/usr/bin/env bash
# SPDX-License-Identifier: MIT
+# shellcheck disable=SC2086 # we want word splitting
+# shellcheck disable=SC1091 # paths only become valid at runtime
-set -e
-set -x
+# If we run in the fork (not from mesa or Marge-bot), reuse mainline kernel and rootfs, if exist.
+_check_artifact_path() {
+ _url="https://${1}/${2}"
+ if curl -s -o /dev/null -I -L -f --retry 4 --retry-delay 15 "${_url}"; then
+ echo -n "${_url}"
+ fi
+}
-# Try to use the kernel and rootfs built in mainline first, so we're more
-# likely to hit cache
-if curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s "https://${BASE_SYSTEM_MAINLINE_HOST_PATH}/done"; then
- BASE_SYSTEM_HOST_PATH="${BASE_SYSTEM_MAINLINE_HOST_PATH}"
-else
- BASE_SYSTEM_HOST_PATH="${BASE_SYSTEM_FORK_HOST_PATH}"
-fi
+get_path_to_artifact() {
+ _mainline_artifact="$(_check_artifact_path ${BASE_SYSTEM_MAINLINE_HOST_PATH} ${1})"
+ if [ -n "${_mainline_artifact}" ]; then
+ echo -n "${_mainline_artifact}"
+ return
+ fi
+ _fork_artifact="$(_check_artifact_path ${BASE_SYSTEM_FORK_HOST_PATH} ${1})"
+ if [ -n "${_fork_artifact}" ]; then
+ echo -n "${_fork_artifact}"
+ return
+ fi
+ set +x
+ error "Sorry, I couldn't find a viable built path for ${1} in either mainline or a fork." >&2
+ echo "" >&2
+ echo "If you're working on CI, this probably means that you're missing a dependency:" >&2
+ echo "this job ran ahead of the job which was supposed to upload that artifact." >&2
+ echo "" >&2
+ echo "If you aren't working on CI, please ping @mesa/ci-helpers to see if we can help." >&2
+ echo "" >&2
+ echo "This job is going to fail, because I can't find the resources I need. Sorry." >&2
+ set -x
+ exit 1
+}
+
+. "${SCRIPTS_DIR}/setup-test-env.sh"
+
+section_start prepare_rootfs "Preparing root filesystem"
+
+set -ex
+
+section_switch rootfs "Assembling root filesystem"
+ROOTFS_URL="$(get_path_to_artifact lava-rootfs.tar.zst)"
+[ $? != 1 ] || exit 1
rm -rf results
mkdir -p results/job-rootfs-overlay/
-cp artifacts/ci-common/capture-devcoredump.sh results/job-rootfs-overlay/
+artifacts/ci-common/generate-env.sh > results/job-rootfs-overlay/set-job-env-vars.sh
cp artifacts/ci-common/init-*.sh results/job-rootfs-overlay/
-cp artifacts/ci-common/intel-gpu-freq.sh results/job-rootfs-overlay/
cp "$SCRIPTS_DIR"/setup-test-env.sh results/job-rootfs-overlay/
-# Prepare env vars for upload.
-section_start variables "Variables passed through:"
-KERNEL_IMAGE_BASE="https://${BASE_SYSTEM_HOST_PATH}" \
- artifacts/ci-common/generate-env.sh | tee results/job-rootfs-overlay/set-job-env-vars.sh
-section_end variables
-
tar zcf job-rootfs-overlay.tar.gz -C results/job-rootfs-overlay/ .
ci-fairy s3cp --token-file "${S3_JWT_FILE}" job-rootfs-overlay.tar.gz "https://${JOB_ROOTFS_OVERLAY_PATH}"
+# Prepare env vars for upload.
+section_switch variables "Environment variables passed through to device:"
+cat results/job-rootfs-overlay/set-job-env-vars.sh
+
+section_switch lava_submit "Submitting job for scheduling"
+
touch results/lava.log
tail -f results/lava.log &
-
PYTHONPATH=artifacts/ artifacts/lava/lava_job_submitter.py \
- submit \
+ --farm "${FARM}" \
+ --device-type "${DEVICE_TYPE}" \
+ --boot-method "${BOOT_METHOD}" \
+ --job-timeout-min $((CI_JOB_TIMEOUT/60 - 5)) \
--dump-yaml \
--pipeline-info "$CI_JOB_NAME: $CI_PIPELINE_URL on $CI_COMMIT_REF_NAME ${CI_NODE_INDEX}/${CI_NODE_TOTAL}" \
- --rootfs-url-prefix "https://${BASE_SYSTEM_HOST_PATH}" \
+ --rootfs-url "${ROOTFS_URL}" \
--kernel-url-prefix "https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}" \
- --build-url "${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/kernel-files.tar.zst" \
- --job-rootfs-overlay-url "${FDO_HTTP_CACHE_URI:-}https://${JOB_ROOTFS_OVERLAY_PATH}" \
- --job-timeout-min ${JOB_TIMEOUT:-80} \
+ --kernel-external "${EXTERNAL_KERNEL_TAG}" \
--first-stage-init artifacts/ci-common/init-stage1.sh \
- --ci-project-dir "${CI_PROJECT_DIR}" \
- --device-type "${DEVICE_TYPE}" \
- --farm "${FARM}" \
--dtb-filename "${DTB}" \
--jwt-file "${S3_JWT_FILE}" \
--kernel-image-name "${KERNEL_IMAGE_NAME}" \
--kernel-image-type "${KERNEL_IMAGE_TYPE}" \
- --boot-method "${BOOT_METHOD}" \
--visibility-group "${VISIBILITY_GROUP}" \
--lava-tags "${LAVA_TAGS}" \
--mesa-job-name "$CI_JOB_NAME" \
--structured-log-file "results/lava_job_detail.json" \
--ssh-client-image "${LAVA_SSH_CLIENT_IMAGE}" \
+ --project-name "${CI_PROJECT_NAME}" \
+ --starting-section "${CURRENT_SECTION}" \
+ --job-submitted-at "${CI_JOB_STARTED_AT}" \
+ - append-overlay \
+ --name=kernel-build \
+ --url="${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/kernel-files.tar.zst" \
+ --compression=zstd \
+ --path="${CI_PROJECT_DIR}" \
+ --format=tar \
+ - append-overlay \
+ --name=job-overlay \
+ --url="https://${JOB_ROOTFS_OVERLAY_PATH}" \
+ --compression=gz \
+ --path="/" \
+ --format=tar \
+ - submit \
>> results/lava.log
diff --git a/drivers/gpu/drm/ci/test.yml b/drivers/gpu/drm/ci/test.yml
index f0ef60c8f56d..c5c786bf419b 100644
--- a/drivers/gpu/drm/ci/test.yml
+++ b/drivers/gpu/drm/ci/test.yml
@@ -1,21 +1,21 @@
.test-rules:
rules:
- - if: '$FD_FARM == "offline" && $RUNNER_TAG =~ /^google-freedreno-/'
- when: never
- - if: '$COLLABORA_FARM == "offline" && $RUNNER_TAG =~ /^mesa-ci-x86-64-lava-/'
- when: never
- !reference [.no_scheduled_pipelines-rules, rules]
+ - !reference [.collabora-farm-rules, rules]
+ - !reference [.google-freedreno-farm-rules, rules]
- when: on_success
.lava-test:
extends:
- .test-rules
+ - .build-rules
+ - .container+build-rules
timeout: "1h30m"
script:
# Note: Build dir (and thus install) may be dirty due to GIT_STRATEGY
- rm -rf install
- tar -xf artifacts/install.tar
- - mv install/* artifacts/.
+ - mv -n install/* artifacts/.
# Override it with our lava-submit.sh script
- ./artifacts/lava-submit.sh
@@ -32,6 +32,7 @@
- alpine/x86_64_lava_ssh_client
- kernel+rootfs_arm32
- debian/x86_64_build
+ - python-artifacts
- testing:arm32
- igt:arm32
@@ -48,6 +49,7 @@
- alpine/x86_64_lava_ssh_client
- kernel+rootfs_arm64
- debian/x86_64_build
+ - python-artifacts
- testing:arm64
- igt:arm64
@@ -64,6 +66,7 @@
- alpine/x86_64_lava_ssh_client
- kernel+rootfs_x86_64
- debian/x86_64_build
+ - python-artifacts
- testing:x86_64
- igt:x86_64
@@ -453,8 +456,6 @@ virtio_gpu:none:
script:
- ln -sf $CI_PROJECT_DIR/install /install
- mv install/bzImage /lava-files/bzImage
- - mkdir -p $CI_PROJECT_DIR/results
- - ln -sf $CI_PROJECT_DIR/results /results
- install/crosvm-runner.sh install/igt_runner.sh
needs:
- debian/x86_64_test-gl
@@ -476,8 +477,6 @@ vkms:none:
- ln -sf $CI_PROJECT_DIR/install /install
- mv install/bzImage /lava-files/bzImage
- mkdir -p /lib/modules
- - mkdir -p $CI_PROJECT_DIR/results
- - ln -sf $CI_PROJECT_DIR/results /results
- ./install/crosvm-runner.sh ./install/igt_runner.sh
needs:
- debian/x86_64_test-gl
--
2.43.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH v2 2/3] drm/ci: update gitlab rules
2025-02-05 8:16 [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Vignesh Raman
2025-02-05 8:16 ` [PATCH v2 1/3] drm/ci: uprev mesa Vignesh Raman
@ 2025-02-05 8:16 ` Vignesh Raman
2025-02-05 8:16 ` [PATCH v2 3/3] drm/ci: update expectation files Vignesh Raman
2025-02-05 14:11 ` [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Helen Mae Koike Fornazier
3 siblings, 0 replies; 5+ messages in thread
From: Vignesh Raman @ 2025-02-05 8:16 UTC (permalink / raw)
To: dri-devel
Cc: daniels, helen.koike, airlied, simona.vetter, robdclark,
guilherme.gallo, sergi.blanch.torne, valentine.burley,
jani.nikula, dmitry.baryshkov, mripard, linux-kernel
Update gitlab rules to include scheduled pipelines.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
---
v2:
- No changes.
---
drivers/gpu/drm/ci/build.yml | 1 -
drivers/gpu/drm/ci/gitlab-ci.yml | 116 +++++++++++++------------------
drivers/gpu/drm/ci/test.yml | 26 +++----
3 files changed, 63 insertions(+), 80 deletions(-)
diff --git a/drivers/gpu/drm/ci/build.yml b/drivers/gpu/drm/ci/build.yml
index a2baa00a20db..274f118533a7 100644
--- a/drivers/gpu/drm/ci/build.yml
+++ b/drivers/gpu/drm/ci/build.yml
@@ -1,6 +1,5 @@
.build:
extends:
- - .build-rules
- .container+build-rules
stage: build-only
artifacts:
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
index 48ab2de9c90b..f04aabe8327c 100644
--- a/drivers/gpu/drm/ci/gitlab-ci.yml
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -41,6 +41,7 @@ variables:
# Python scripts for structured logger
PYTHONPATH: "$PYTHONPATH:$CI_PROJECT_DIR/install"
+
default:
id_tokens:
S3_JWT:
@@ -67,6 +68,7 @@ default:
export S3_JWT="$(<${S3_JWT_FILE})" &&
rm "${S3_JWT_FILE}"
+
include:
- project: 'freedesktop/ci-templates'
ref: 16bc29078de5e0a067ff84a1a199a3760d3b3811
@@ -126,33 +128,27 @@ stages:
- rockchip
- software-driver
+
# YAML anchors for rule conditions
# --------------------------------
.rules-anchors:
rules:
- # Pipeline for forked project branch
- - if: &is-forked-branch '$CI_COMMIT_BRANCH && $CI_PROJECT_NAMESPACE != "mesa"'
- when: manual
- # Forked project branch / pre-merge pipeline not for Marge bot
- - if: &is-forked-branch-or-pre-merge-not-for-marge '$CI_PROJECT_NAMESPACE != "mesa" || ($GITLAB_USER_LOGIN != "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event")'
- when: manual
- # Pipeline runs for the main branch of the upstream Mesa project
- - if: &is-mesa-main '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH'
- when: always
- # Post-merge pipeline
- - if: &is-post-merge '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_BRANCH'
- when: on_success
- # Post-merge pipeline, not for Marge Bot
- - if: &is-post-merge-not-for-marge '$CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot" && $CI_COMMIT_BRANCH'
- when: on_success
+ # do not duplicate pipelines on merge pipelines
+ - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
+ when: never
+ # merge pipeline
+ - if: &is-merge-attempt $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"
+ # post-merge pipeline
+ - if: &is-post-merge $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "push"
# Pre-merge pipeline
- - if: &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
- when: on_success
- # Pre-merge pipeline for Marge Bot
- - if: &is-pre-merge-for-marge '$GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"'
- when: on_success
+ - if: &is-pre-merge $CI_PIPELINE_SOURCE == "merge_request_event"
# Push to a branch on a fork
- - &is-fork-push '$CI_PROJECT_NAMESPACE != "mesa" && $CI_PIPELINE_SOURCE == "push"'
+ - if: &is-fork-push $CI_PROJECT_NAMESPACE != "mesa" && $CI_PIPELINE_SOURCE == "push"
+ # nightly pipeline
+ - if: &is-scheduled-pipeline $CI_PIPELINE_SOURCE == "schedule"
+ # pipeline for direct pushes that bypassed the CI
+ - if: &is-direct-push $CI_PROJECT_NAMESPACE == "mesa" && $CI_PIPELINE_SOURCE == "push" && $GITLAB_USER_LOGIN != "marge-bot"
+
# Rules applied to every job in the pipeline
.common-rules:
@@ -160,39 +156,48 @@ stages:
- if: *is-fork-push
when: manual
+
.never-post-merge-rules:
rules:
- if: *is-post-merge
when: never
-# Rule to filter for only scheduled pipelines.
-.scheduled_pipeline-rules:
- rules:
- - if: &is-scheduled-pipeline '$CI_PIPELINE_SOURCE == "schedule"'
- when: on_success
-
-# Generic rule to not run the job during scheduled pipelines. Jobs that aren't
-# something like a nightly run should include this rule.
-.no_scheduled_pipelines-rules:
- rules:
- - if: *is-scheduled-pipeline
- when: never
-# When to automatically run the CI for build jobs
-.build-rules:
- rules:
- - !reference [.no_scheduled_pipelines-rules, rules]
- - !reference [.never-post-merge-rules, rules]
- # Run automatically once all dependency jobs have passed
- - when: on_success
-
-# When to automatically run the CI for container jobs
.container+build-rules:
rules:
- - !reference [.no_scheduled_pipelines-rules, rules]
+ - !reference [.common-rules, rules]
+ # Run when re-enabling a disabled farm, but not when disabling it
+ - !reference [.disable-farm-mr-rules, rules]
+ # Never run immediately after merging, as we just ran everything
- !reference [.never-post-merge-rules, rules]
+ # Build everything in merge pipelines, if any files affecting the pipeline
+ # were changed
+ - if: *is-merge-attempt
+ changes: &all_paths
+ - drivers/gpu/drm/ci/**/*
+ when: on_success
+ # Same as above, but for pre-merge pipelines
+ - if: *is-pre-merge
+ changes:
+ *all_paths
+ when: manual
+ # Skip everything for pre-merge and merge pipelines which don't change
+ # anything in the build
+ - if: *is-merge-attempt
+ when: never
+ - if: *is-pre-merge
+ when: never
+ # Build everything after someone bypassed the CI
+ - if: *is-direct-push
+ when: on_success
+ # Build everything in scheduled pipelines
+ - if: *is-scheduled-pipeline
+ when: on_success
+ # Allow building everything in fork pipelines, but build nothing unless
+ # manually triggered
- when: manual
+
.ci-deqp-artifacts:
artifacts:
name: "${CI_PROJECT_NAME}_${CI_JOB_NAME}"
@@ -206,31 +211,7 @@ stages:
- _build/meson-logs/strace
-.container-rules:
- rules:
- - !reference [.no_scheduled_pipelines-rules, rules]
- - !reference [.never-post-merge-rules, rules]
- # Run pipeline by default in the main project if any CI pipeline
- # configuration files were changed, to ensure docker images are up to date
- - if: *is-post-merge
- changes:
- - drivers/gpu/drm/ci/**/*
- when: on_success
- # Run pipeline by default if it was triggered by Marge Bot, is for a
- # merge request, and any files affecting the pipeline were changed
- - if: *is-pre-merge-for-marge
- when: on_success
- # Run pipeline by default in the main project if it was not triggered by
- # Marge Bot, and any files affecting the pipeline were changed
- - if: *is-post-merge-not-for-marge
- when: on_success
- # Allow triggering jobs manually in other cases
- - when: manual
-
-
-
# Git archive
-
make git archive:
extends:
- .fdo.ci-fairy
@@ -313,6 +294,7 @@ mr-label-maker-test:
- pip install git+https://gitlab.freedesktop.org/freedesktop/mr-label-maker
- mr-label-maker --dry-run --mr $CI_MERGE_REQUEST_IID
+
# Jobs that need to pass before spending hardware resources on further testing
.required-for-hardware-jobs:
needs:
diff --git a/drivers/gpu/drm/ci/test.yml b/drivers/gpu/drm/ci/test.yml
index c5c786bf419b..6a1e059858e5 100644
--- a/drivers/gpu/drm/ci/test.yml
+++ b/drivers/gpu/drm/ci/test.yml
@@ -1,16 +1,11 @@
-.test-rules:
- rules:
- - !reference [.no_scheduled_pipelines-rules, rules]
- - !reference [.collabora-farm-rules, rules]
- - !reference [.google-freedreno-farm-rules, rules]
- - when: on_success
-
.lava-test:
extends:
- - .test-rules
- - .build-rules
- .container+build-rules
timeout: "1h30m"
+ rules:
+ - !reference [.scheduled_pipeline-rules, rules]
+ - !reference [.collabora-farm-rules, rules]
+ - when: on_success
script:
# Note: Build dir (and thus install) may be dirty due to GIT_STRATEGY
- rm -rf install
@@ -74,8 +69,11 @@
extends:
- .baremetal-test-arm64
- .use-debian/baremetal_arm64_test
- - .test-rules
timeout: "1h30m"
+ rules:
+ - !reference [.scheduled_pipeline-rules, rules]
+ - !reference [.google-freedreno-farm-rules, rules]
+ - when: on_success
variables:
FDO_CI_CONCURRENT: 10
HWCI_TEST_SCRIPT: "/install/igt_runner.sh"
@@ -444,13 +442,15 @@ panfrost:g12b:
virtio_gpu:none:
stage: software-driver
timeout: "1h30m"
+ rules:
+ - !reference [.scheduled_pipeline-rules, rules]
+ - when: on_success
variables:
CROSVM_GALLIUM_DRIVER: llvmpipe
DRIVER_NAME: virtio_gpu
GPU_VERSION: none
extends:
- .test-gl
- - .test-rules
tags:
- kvm
script:
@@ -465,12 +465,14 @@ virtio_gpu:none:
vkms:none:
stage: software-driver
timeout: "1h30m"
+ rules:
+ - !reference [.scheduled_pipeline-rules, rules]
+ - when: on_success
variables:
DRIVER_NAME: vkms
GPU_VERSION: none
extends:
- .test-gl
- - .test-rules
tags:
- kvm
script:
--
2.43.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH v2 3/3] drm/ci: update expectation files
2025-02-05 8:16 [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Vignesh Raman
2025-02-05 8:16 ` [PATCH v2 1/3] drm/ci: uprev mesa Vignesh Raman
2025-02-05 8:16 ` [PATCH v2 2/3] drm/ci: update gitlab rules Vignesh Raman
@ 2025-02-05 8:16 ` Vignesh Raman
2025-02-05 14:11 ` [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Helen Mae Koike Fornazier
3 siblings, 0 replies; 5+ messages in thread
From: Vignesh Raman @ 2025-02-05 8:16 UTC (permalink / raw)
To: dri-devel
Cc: daniels, helen.koike, airlied, simona.vetter, robdclark,
guilherme.gallo, sergi.blanch.torne, valentine.burley,
jani.nikula, dmitry.baryshkov, mripard, linux-kernel
Update expectation files for the mesa uprev.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
---
v2:
- New commit in v2.
---
drivers/gpu/drm/ci/xfails/vkms-none-fails.txt | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt b/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt
index 85647eca4e55..6ebcc7d89fbd 100644
--- a/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt
+++ b/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt
@@ -13,6 +13,8 @@ kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
kms_cursor_legacy@flip-vs-cursor-legacy,Fail
kms_flip@flip-vs-modeset-vs-hang,Fail
kms_flip@flip-vs-panning-vs-hang,Fail
+kms_flip@flip-vs-suspend,Fail
+kms_flip@flip-vs-suspend-interruptible,Fail
kms_lease@lease-uevent,Fail
kms_writeback@writeback-check-output,Fail
kms_writeback@writeback-check-output-XRGB2101010,Fail
--
2.43.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules
2025-02-05 8:16 [PATCH v2 0/3] drm/ci: uprev mesa and modify gitlab rules Vignesh Raman
` (2 preceding siblings ...)
2025-02-05 8:16 ` [PATCH v2 3/3] drm/ci: update expectation files Vignesh Raman
@ 2025-02-05 14:11 ` Helen Mae Koike Fornazier
3 siblings, 0 replies; 5+ messages in thread
From: Helen Mae Koike Fornazier @ 2025-02-05 14:11 UTC (permalink / raw)
To: Vignesh Raman
Cc: dri-devel, daniels, airlied, simona.vetter, robdclark,
guilherme.gallo, sergi.blanch.torne, valentine.burley,
jani.nikula, dmitry.baryshkov, mripard, linux-kernel
---- On Wed, 05 Feb 2025 05:16:45 -0300 Vignesh Raman wrote ---
> Uprev mesa to adapt to the latest changes in mesa-ci,
> including new container jobs and stages. Update the
> lava-submit script to align with recent mes-ci changes
> for using LAVA rootfs overlays. Modify gitLab rules
> to include scheduled pipelines. Also update the
> expectation files.
>
> Pipeline link,
> https://gitlab.freedesktop.org/vigneshraman/linux/-/pipelines/1358207
>
> Link to v1,
> https://lore.kernel.org/all/20241128042025.611659-1-vignesh.raman@collabora.com/
>
> Vignesh Raman (2):
> drm/ci: uprev mesa
> drm/ci: update gitlab rules
> drm/ci: update expectation files
>
> drivers/gpu/drm/ci/build.sh | 2 +-
> drivers/gpu/drm/ci/build.yml | 104 +++++++++-
> drivers/gpu/drm/ci/container.yml | 22 +-
> drivers/gpu/drm/ci/gitlab-ci.yml | 195 ++++++++++--------
> drivers/gpu/drm/ci/igt_runner.sh | 13 +-
> drivers/gpu/drm/ci/image-tags.yml | 11 +-
> drivers/gpu/drm/ci/lava-submit.sh | 100 ++++++---
> drivers/gpu/drm/ci/test.yml | 37 ++--
> drivers/gpu/drm/ci/xfails/vkms-none-fails.txt | 2 +
> 9 files changed, 325 insertions(+), 161 deletions(-)
>
> --
> 2.43.0
>
>
Applied to drm-misc-next
Thanks
Helen
^ permalink raw reply [flat|nested] 5+ messages in thread