From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4EA021E091 for ; Sat, 11 Apr 2026 02:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.180.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775874173; cv=none; b=WNjck7lSX/WQL5tfQapkv6PCJcKPOMCSAIE2ovt9W7xIarvfuYLhOO5A+I5pJhz0yJo40yJVp5cWMXOuRHlEl8ud85+akF6l9vDQquA5xNpfbhpIGXk4IxBquPr2CA2Z913l0O8Ttd01lCMiUdNd2zLu9XB3ErEa/VkJCckhRwY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775874173; c=relaxed/simple; bh=cjlhZdPE0y3zOZPRXjtMrt/oUgujMYDJ/vFmReFGNBU=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=cIVJdVtmwzxpPC2Gt9qJenWW1TnUOmDEhBcJqS8WSGlF070flosBwnw1+d52eEWoX2OcfR+iZEiR/rqmFAAZZABurnStT9gXyvBr9E/eAfo17QvWiE+41iq1QTqUNOliAYtQV3GeHFas1oi31UQX0waQtdhwfyZYQ7HsbBT3SkU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=QQW8pBiI; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=b42c8kbI; arc=none smtp.client-ip=205.220.180.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="QQW8pBiI"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="b42c8kbI" Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 63AFJOLn3118612 for ; Sat, 11 Apr 2026 02:22:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= jsPK59L1DEHDhwsLvGu/62voFLqLbdR2wUTVJKE17k8=; b=QQW8pBiI8qFZn7n8 UVGjw6oKqIKXC7MqB9kSiym8WuRWcNEwfw8MZqMDKgv0dnumYoKqZjXUL9SBHgc8 WdU2RUwoWZ45S4FaM729VQDpBfID14tqowKSBxDnhL82tZK7GLZyMBz2Zk4HPvgE LpViPigUowPCaUuioatfKTAePWeerB+JqH/jHRj5ZXQKg9RuXcQiFGCUAy3XROHL 9SAjAzIn1vmqt6YZ3q4C0mkeBTNYm2rS4NjRCp4s2BE3WDpf/gURUQjGu/dCfBHt CMYqDoDqMdUtDeBPCc5yq62nhZOPvpukuV/9rRBPX44ta4MsIkco6V+p7r+Kapqs hNJvQQ== Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4df3rasguf-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Sat, 11 Apr 2026 02:22:50 +0000 (GMT) Received: by mail-pl1-f199.google.com with SMTP id d9443c01a7336-2b24e45271cso28828055ad.2 for ; Fri, 10 Apr 2026 19:22:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1775874169; x=1776478969; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=jsPK59L1DEHDhwsLvGu/62voFLqLbdR2wUTVJKE17k8=; b=b42c8kbIdPbJ4uetEdcbO3grlKH3u6E2e8a8oZFi8SlS+Me/0zi/Fy8uLdk4+WR/w1 CilyiFBqZ83Mq11KG5WHkk03oACqZ/eszMnXVRzq8dLGgQPMS4nn7BhZlRBHLOrmUj5t tmkY2DH2QlMaIJf9fs8mqdSR3A3WX3Sm0vzD1Apnv0+GSfyhshH/Bh2NlprjcG0kKTNn PEl9X/7OnBQEfE4I82VqmfqD1v4kwtQC3VcNHcKEyCQYYrPHfPcVN/aXO+kh7uQ25SrQ A1RXT7h3W8Ax2WgUWUSXTchNCl10qR8K0ROSJHsdwS/5C9gZDLdM6jq02baUGWDCXwP2 N2Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775874169; x=1776478969; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jsPK59L1DEHDhwsLvGu/62voFLqLbdR2wUTVJKE17k8=; b=UAyDh85Nf039dkrYLq+b0DIg8DTPJzKty1fv8tJ4wtMVeJDbVjaUc1+XtIl6XuKyvs GMIqCH2guZH9jJYsCZBmJdmP75GDTl0tv8/rEDU879ia0r0rnS72BvU/60tRA10bCQSf /XmRrfkwELGm/Ic7an3d9+PtTAMOAt1B6VKXXIAo/RddQdzJ4iYtbq78CY3j1XFQtzq5 O1lxJwaI5OERdQljFapY+4uoYhsugSMG9YhsVa3monKcWiBT123lqOCkqIqlbjDV5yua cLdvTEPn3B6R6z0zCmp6fLBkUVDd63PwyxXPx2p9eBlg2LzqM9XFJtKw9OZdX7y9O4LW tYFg== X-Forwarded-Encrypted: i=1; AJvYcCXjVR5g6ChnegsVyhNncitipXQJ8c9/WP211ofUeknAs4aVyBpQKWb+iyIrRAJ/hhnK0H6+MyrQNeWjb6E=@vger.kernel.org X-Gm-Message-State: AOJu0YyDyHc9o21TDnYxtZ42pZQaFAzwc1IVeIUAvXLNnnu24xnNEgsX dc82AbhF0m7MORdECGt0AJ+snMP8qWuF1oRyqsNlJRbUZBKY+SvyoGuelvVZu3DM6+iHrh6YWrd 5F4DH+U9XfXhdj/gFb+S8WjwnIxkh+BsPVcF+oslrH/Wd0VlwEAyoe4hvPpmQXb4auDo= X-Gm-Gg: AeBDietO7FJF27SoX5QFJRSws2E3+WRrKq7DGWH9a02Cr8cXKTdH5g21Kvi9cZZGzJ9 4178dBsYHH4uNjHApcI5MHOxJKxikn9XqJvXGL+hyU6UIwiZXLDnlsMNCuvP6rQuw5wltyc86Ao KklFrb/xtmTuvyUCPJ5nE2Gu7aT/afVWxCR1NRBigdunwhJYmKmPbudwUp3oiJ0BjW5D6k3saNA xnsiEbyLY+OvOXoBF68jBV4etLMEtEjD93rmPkCkXfyaRwuP+hx4l1w9Oeptu6uFktCE+sNOGHx Yfa2Vf0/8y3ijkyCIpgv8b8NHni9g8MlHZAHoJVq0BRIOEwqyHCICnQtX34T98JSpiHuNVyC3VP Hn02fm23H1A2I7GYUr12V4hxCgO3VgDSam840Ah9S/PdtHSlMrT1JxWl+NI7ONj4CDYYIl0kLag GTqeJLUzp3xg== X-Received: by 2002:a17:902:ffcb:b0:2ae:aa16:acfb with SMTP id d9443c01a7336-2b2d5a304b9mr60539135ad.22.1775874169321; Fri, 10 Apr 2026 19:22:49 -0700 (PDT) X-Received: by 2002:a17:902:ffcb:b0:2ae:aa16:acfb with SMTP id d9443c01a7336-2b2d5a304b9mr60538885ad.22.1775874168671; Fri, 10 Apr 2026 19:22:48 -0700 (PDT) Received: from [10.133.33.83] (tpe-colo-wan-fw-bordernet.qualcomm.com. [103.229.16.4]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b2d4f3afebsm42714125ad.72.2026.04.10.19.22.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Apr 2026 19:22:48 -0700 (PDT) Message-ID: <07fcf759-b53c-4bc8-8e88-48b82713fecb@oss.qualcomm.com> Date: Sat, 11 Apr 2026 10:22:41 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 15/17] drm/msm/a8xx: Preemption support for A840 To: Akhil P Oommen , Rob Clark , Sean Paul , Konrad Dybcio , Dmitry Baryshkov , Abhinav Kumar , Jessica Zhang , Marijn Suijten , David Airlie , Simona Vetter , Antonino Maniscalco , Connor Abbott , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org References: <20260327-a8xx-gpu-batch2-v2-0-2b53c38d2101@oss.qualcomm.com> <20260327-a8xx-gpu-batch2-v2-15-2b53c38d2101@oss.qualcomm.com> Content-Language: en-US From: Jie Gan In-Reply-To: <20260327-a8xx-gpu-batch2-v2-15-2b53c38d2101@oss.qualcomm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authority-Analysis: v=2.4 cv=Vs4Txe2n c=1 sm=1 tr=0 ts=69d9b07a cx=c_pps a=JL+w9abYAAE89/QcEU+0QA==:117 a=nuhDOHQX5FNHPW3J6Bj6AA==:17 a=IkcTkHD0fZMA:10 a=A5OVakUREuEA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=ZpdpYltYx_vBUK5n70dp:22 a=EUspDBNiAAAA:8 a=68z4iRH7ockWEGeBAzMA:9 a=QEXdDO2ut3YA:10 a=O8hF6Hzn-FEA:10 a=324X-CrmTo6CU4MGRt3R:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDExMDAxNiBTYWx0ZWRfX8itjMjoa72/L p7n/f8UM3efIw4nfrolTM48UF3g6yXKrWG6l1CJbH2rDh20aS8QxvILdJiGyRRJWlfjrMvSsCBJ aKZQPeosWgkOcUfWtNVyCb47i0MHoN7gmq8J2v+djsCD/P08t/czIkg41rbNdR69SZVocJWK/6I BB1U12Oeyefm8YVf2ymHNBt34XYo9o4fptUawWrMobR+MOMMW8ZuHWCkwkcWrWMITXbnrStQexq xn6OIoMpnKsng9mlFa0wst3uC35rQFo4LOlcZpEO9pO2PTMLJwwD1sXB9FM91Wa4naQQK3hltPZ 4Y9srtZnJF6MSSFTolnpyaxvQNhJltx4SCkazKov9hqeSqQ+GcnCn/FEEQa0iTsQANGUBBXY/K6 ZdDMYNCtQOdm4uJ448tnw+HHIKWnuHXp5aQrdNIshJxJDptC+gtbNXlHgqRmflvqJdSmalNsRgY erpZRJWnJtC4jVUx75A== X-Proofpoint-ORIG-GUID: Mgoc5vjFUOEdzco_E4Xpoagx1Yz1TnRA X-Proofpoint-GUID: Mgoc5vjFUOEdzco_E4Xpoagx1Yz1TnRA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-10_07,2026-04-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 adultscore=0 malwarescore=0 bulkscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 phishscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2604010000 definitions=main-2604110016 On 3/27/2026 8:14 AM, Akhil P Oommen wrote: > The programing sequence related to preemption is unchanged from A7x. But > there is some code churn due to register shuffling in A8x. So, split out > the common code into a header file for code sharing and add/update > additional changes required to support preemption feature on A8x GPUs. > > Finally, enable the preemption quirk in A840's catalog to enable this > feature. > > Signed-off-by: Akhil P Oommen > --- > drivers/gpu/drm/msm/Makefile | 1 + > drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 1 + > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 7 +- > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 5 + > drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 77 +-------- > drivers/gpu/drm/msm/adreno/a6xx_preempt.h | 82 ++++++++++ > drivers/gpu/drm/msm/adreno/a8xx_gpu.c | 37 ++++- > drivers/gpu/drm/msm/adreno/a8xx_preempt.c | 259 ++++++++++++++++++++++++++++++ > 8 files changed, 389 insertions(+), 80 deletions(-) > > diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile > index 8b94c5f1cb68..ba45e99be05b 100644 > --- a/drivers/gpu/drm/msm/Makefile > +++ b/drivers/gpu/drm/msm/Makefile > @@ -25,6 +25,7 @@ adreno-y := \ > adreno/a6xx_hfi.o \ > adreno/a6xx_preempt.o \ > adreno/a8xx_gpu.o \ > + adreno/a8xx_preempt.o \ > > adreno-$(CONFIG_DEBUG_FS) += adreno/a5xx_debugfs.o \ > > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c > index 53548f6e891b..21f5a685196b 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c > +++ b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c > @@ -2120,6 +2120,7 @@ static const struct adreno_info a8xx_gpus[] = { > .inactive_period = DRM_MSM_INACTIVE_PERIOD, > .quirks = ADRENO_QUIRK_HAS_CACHED_COHERENT | > ADRENO_QUIRK_HAS_HW_APRIV | > + ADRENO_QUIRK_PREEMPTION | > ADRENO_QUIRK_IFPC, > .funcs = &a8xx_gpu_funcs, > .a6xx = &(const struct a6xx_info) { > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c > index fb9662b946d0..44ce02b412ca 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c > @@ -410,7 +410,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) > a6xx_flush(gpu, ring); > } > > -static void a6xx_emit_set_pseudo_reg(struct msm_ringbuffer *ring, > +void a6xx_emit_set_pseudo_reg(struct msm_ringbuffer *ring, > struct a6xx_gpu *a6xx_gpu, struct msm_gpu_submitqueue *queue) > { > u64 preempt_postamble; > @@ -620,7 +620,10 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) > a6xx_flush(gpu, ring); > > /* Check to see if we need to start preemption */ > - a6xx_preempt_trigger(gpu); > + if (adreno_is_a8xx(adreno_gpu)) > + a8xx_preempt_trigger(gpu); > + else > + a6xx_preempt_trigger(gpu); > } > > static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > index a4434a6a56dd..eb431e5e00b1 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > @@ -278,6 +278,8 @@ void a6xx_preempt_hw_init(struct msm_gpu *gpu); > void a6xx_preempt_trigger(struct msm_gpu *gpu); > void a6xx_preempt_irq(struct msm_gpu *gpu); > void a6xx_preempt_fini(struct msm_gpu *gpu); > +void a6xx_emit_set_pseudo_reg(struct msm_ringbuffer *ring, > + struct a6xx_gpu *a6xx_gpu, struct msm_gpu_submitqueue *queue); > int a6xx_preempt_submitqueue_setup(struct msm_gpu *gpu, > struct msm_gpu_submitqueue *queue); > void a6xx_preempt_submitqueue_close(struct msm_gpu *gpu, > @@ -327,6 +329,9 @@ void a8xx_gpu_get_slice_info(struct msm_gpu *gpu); > int a8xx_hw_init(struct msm_gpu *gpu); > irqreturn_t a8xx_irq(struct msm_gpu *gpu); > void a8xx_llc_activate(struct a6xx_gpu *a6xx_gpu); > +void a8xx_preempt_hw_init(struct msm_gpu *gpu); > +void a8xx_preempt_trigger(struct msm_gpu *gpu); > +void a8xx_preempt_irq(struct msm_gpu *gpu); > bool a8xx_progress(struct msm_gpu *gpu, struct msm_ringbuffer *ring); > void a8xx_recover(struct msm_gpu *gpu); > #endif /* __A6XX_GPU_H__ */ > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c > index 747a22afad9f..df4cbf42e9a4 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c > +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c > @@ -6,85 +6,10 @@ > #include "msm_gem.h" > #include "a6xx_gpu.h" > #include "a6xx_gmu.xml.h" > +#include "a6xx_preempt.h" > #include "msm_mmu.h" > #include "msm_gpu_trace.h" > > -/* > - * Try to transition the preemption state from old to new. Return > - * true on success or false if the original state wasn't 'old' > - */ > -static inline bool try_preempt_state(struct a6xx_gpu *a6xx_gpu, > - enum a6xx_preempt_state old, enum a6xx_preempt_state new) > -{ > - enum a6xx_preempt_state cur = atomic_cmpxchg(&a6xx_gpu->preempt_state, > - old, new); > - > - return (cur == old); > -} > - > -/* > - * Force the preemption state to the specified state. This is used in cases > - * where the current state is known and won't change > - */ > -static inline void set_preempt_state(struct a6xx_gpu *gpu, > - enum a6xx_preempt_state new) > -{ > - /* > - * preempt_state may be read by other cores trying to trigger a > - * preemption or in the interrupt handler so barriers are needed > - * before... > - */ > - smp_mb__before_atomic(); > - atomic_set(&gpu->preempt_state, new); > - /* ... and after*/ > - smp_mb__after_atomic(); > -} > - > -/* Write the most recent wptr for the given ring into the hardware */ > -static inline void update_wptr(struct a6xx_gpu *a6xx_gpu, struct msm_ringbuffer *ring) > -{ > - unsigned long flags; > - uint32_t wptr; > - > - spin_lock_irqsave(&ring->preempt_lock, flags); > - > - if (ring->restore_wptr) { > - wptr = get_wptr(ring); > - > - a6xx_fenced_write(a6xx_gpu, REG_A6XX_CP_RB_WPTR, wptr, BIT(0), false); > - > - ring->restore_wptr = false; > - } > - > - spin_unlock_irqrestore(&ring->preempt_lock, flags); > -} > - > -/* Return the highest priority ringbuffer with something in it */ > -static struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) > -{ > - struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > - struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > - > - unsigned long flags; > - int i; > - > - for (i = 0; i < gpu->nr_rings; i++) { > - bool empty; > - struct msm_ringbuffer *ring = gpu->rb[i]; > - > - spin_lock_irqsave(&ring->preempt_lock, flags); > - empty = (get_wptr(ring) == gpu->funcs->get_rptr(gpu, ring)); > - if (!empty && ring == a6xx_gpu->cur_ring) > - empty = ring->memptrs->fence == a6xx_gpu->last_seqno[i]; > - spin_unlock_irqrestore(&ring->preempt_lock, flags); > - > - if (!empty) > - return ring; > - } > - > - return NULL; > -} > - > static void a6xx_preempt_timer(struct timer_list *t) > { > struct a6xx_gpu *a6xx_gpu = timer_container_of(a6xx_gpu, t, > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.h b/drivers/gpu/drm/msm/adreno/a6xx_preempt.h > new file mode 100644 > index 000000000000..df36c945b836 > --- /dev/null > +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.h > @@ -0,0 +1,82 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* Copyright (c) 2018, The Linux Foundation. All rights reserved. */ > +/* Copyright (c) 2023 Collabora, Ltd. */ > +/* Copyright (c) 2024 Valve Corporation */ > +/* Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. */ > + > +/* > + * Try to transition the preemption state from old to new. Return > + * true on success or false if the original state wasn't 'old' > + */ > +static inline bool try_preempt_state(struct a6xx_gpu *a6xx_gpu, > + enum a6xx_preempt_state old, enum a6xx_preempt_state new) > +{ > + enum a6xx_preempt_state cur = atomic_cmpxchg(&a6xx_gpu->preempt_state, > + old, new); > + > + return (cur == old); > +} > + > +/* > + * Force the preemption state to the specified state. This is used in cases > + * where the current state is known and won't change > + */ > +static inline void set_preempt_state(struct a6xx_gpu *gpu, > + enum a6xx_preempt_state new) > +{ > + /* > + * preempt_state may be read by other cores trying to trigger a > + * preemption or in the interrupt handler so barriers are needed > + * before... > + */ > + smp_mb__before_atomic(); > + atomic_set(&gpu->preempt_state, new); > + /* ... and after */ > + smp_mb__after_atomic(); > +} > + > +/* Write the most recent wptr for the given ring into the hardware */ > +static inline void update_wptr(struct a6xx_gpu *a6xx_gpu, struct msm_ringbuffer *ring) > +{ > + unsigned long flags; > + uint32_t wptr; > + > + spin_lock_irqsave(&ring->preempt_lock, flags); > + > + if (ring->restore_wptr) { > + wptr = get_wptr(ring); > + > + a6xx_fenced_write(a6xx_gpu, REG_A6XX_CP_RB_WPTR, wptr, BIT(0), false); > + > + ring->restore_wptr = false; > + } > + > + spin_unlock_irqrestore(&ring->preempt_lock, flags); > +} > + > +/* Return the highest priority ringbuffer with something in it */ > +static inline struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) > +{ > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + > + unsigned long flags; > + int i; > + > + for (i = 0; i < gpu->nr_rings; i++) { > + bool empty; > + struct msm_ringbuffer *ring = gpu->rb[i]; > + > + spin_lock_irqsave(&ring->preempt_lock, flags); > + empty = (get_wptr(ring) == gpu->funcs->get_rptr(gpu, ring)); > + if (!empty && ring == a6xx_gpu->cur_ring) > + empty = ring->memptrs->fence == a6xx_gpu->last_seqno[i]; > + spin_unlock_irqrestore(&ring->preempt_lock, flags); > + > + if (!empty) > + return ring; > + } > + > + return NULL; > +} > + > diff --git a/drivers/gpu/drm/msm/adreno/a8xx_gpu.c b/drivers/gpu/drm/msm/adreno/a8xx_gpu.c > index e406681b8c80..9e6f2ed69247 100644 > --- a/drivers/gpu/drm/msm/adreno/a8xx_gpu.c > +++ b/drivers/gpu/drm/msm/adreno/a8xx_gpu.c > @@ -463,6 +463,34 @@ static void a8xx_patch_pwrup_reglist(struct msm_gpu *gpu) > a8xx_aperture_clear(gpu); > } > > +static int a8xx_preempt_start(struct msm_gpu *gpu) > +{ > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + struct msm_ringbuffer *ring = gpu->rb[0]; > + > + if (gpu->nr_rings <= 1) > + return 0; > + > + /* Turn CP protection off */ > + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); > + OUT_RING(ring, 0); > + > + a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, NULL); > + > + /* Yield the floor on command completion */ > + OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4); > + OUT_RING(ring, 0x00); > + OUT_RING(ring, 0x00); > + OUT_RING(ring, 0x00); > + /* Generate interrupt on preemption completion */ > + OUT_RING(ring, 0x00); > + > + a6xx_flush(gpu, ring); > + > + return a8xx_idle(gpu, ring) ? 0 : -EINVAL; > +} > + > static int a8xx_cp_init(struct msm_gpu *gpu) > { > struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > @@ -738,6 +766,8 @@ static int hw_init(struct msm_gpu *gpu) > gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR, shadowptr(a6xx_gpu, gpu->rb[0])); > gpu_write64(gpu, REG_A8XX_CP_RB_RPTR_ADDR_BV, rbmemptr(gpu->rb[0], bv_rptr)); > > + a8xx_preempt_hw_init(gpu); > + > for (i = 0; i < gpu->nr_rings; i++) > a6xx_gpu->shadow[i] = 0; > > @@ -800,6 +830,9 @@ static int hw_init(struct msm_gpu *gpu) > /* Enable hardware clockgating */ > a8xx_set_hwcg(gpu, true); > out: > + /* Last step - yield the ringbuffer */ > + a8xx_preempt_start(gpu); > + > /* > * Tell the GMU that we are done touching the GPU and it can start power > * management > @@ -1209,11 +1242,11 @@ irqreturn_t a8xx_irq(struct msm_gpu *gpu) > > if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) { > msm_gpu_retire(gpu); > - a6xx_preempt_trigger(gpu); > + a8xx_preempt_trigger(gpu); > } > > if (status & A6XX_RBBM_INT_0_MASK_CP_SW) > - a6xx_preempt_irq(gpu); > + a8xx_preempt_irq(gpu); > > return IRQ_HANDLED; > } > diff --git a/drivers/gpu/drm/msm/adreno/a8xx_preempt.c b/drivers/gpu/drm/msm/adreno/a8xx_preempt.c > new file mode 100644 > index 000000000000..3d8c33ba722e > --- /dev/null > +++ b/drivers/gpu/drm/msm/adreno/a8xx_preempt.c > @@ -0,0 +1,259 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. */ > + > +#include "msm_gem.h" > +#include "a6xx_gpu.h" > +#include "a6xx_gmu.xml.h" > +#include "a6xx_preempt.h" > +#include "msm_mmu.h" > +#include "msm_gpu_trace.h" > + > +static void preempt_prepare_postamble(struct a6xx_gpu *a6xx_gpu) > +{ > + u32 *postamble = a6xx_gpu->preempt_postamble_ptr; > + u32 count = 0; > + > + postamble[count++] = PKT7(CP_REG_RMW, 3); > + postamble[count++] = REG_A8XX_RBBM_PERFCTR_SRAM_INIT_CMD; > + postamble[count++] = 0; > + postamble[count++] = 1; > + > + postamble[count++] = PKT7(CP_WAIT_REG_MEM, 6); > + postamble[count++] = CP_WAIT_REG_MEM_0_FUNCTION(WRITE_EQ); > + postamble[count++] = CP_WAIT_REG_MEM_POLL_ADDR_LO( > + REG_A8XX_RBBM_PERFCTR_SRAM_INIT_STATUS); > + postamble[count++] = CP_WAIT_REG_MEM_POLL_ADDR_HI(0); > + postamble[count++] = CP_WAIT_REG_MEM_3_REF(0x1); > + postamble[count++] = CP_WAIT_REG_MEM_4_MASK(0x1); > + postamble[count++] = CP_WAIT_REG_MEM_5_DELAY_LOOP_CYCLES(0); > + > + a6xx_gpu->preempt_postamble_len = count; > + > + a6xx_gpu->postamble_enabled = true; > +} > + > +static void preempt_disable_postamble(struct a6xx_gpu *a6xx_gpu) > +{ > + u32 *postamble = a6xx_gpu->preempt_postamble_ptr; > + > + /* > + * Disable the postamble by replacing the first packet header with a NOP > + * that covers the whole buffer. > + */ > + *postamble = PKT7(CP_NOP, (a6xx_gpu->preempt_postamble_len - 1)); > + > + a6xx_gpu->postamble_enabled = false; > +} > + > +/* > + * Set preemption keepalive vote. Please note that this vote is different from the one used in > + * a8xx_irq() > + */ > +static void a8xx_preempt_keepalive_vote(struct msm_gpu *gpu, bool on) > +{ > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + > + gmu_write(&a6xx_gpu->gmu, REG_A8XX_GMU_PWR_COL_PREEMPT_KEEPALIVE, on); > +} > + > +void a8xx_preempt_irq(struct msm_gpu *gpu) > +{ > + uint32_t status; > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + struct drm_device *dev = gpu->dev; > + > + if (!try_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED, PREEMPT_PENDING)) > + return; > + > + /* Delete the preemption watchdog timer */ > + timer_delete(&a6xx_gpu->preempt_timer); > + > + /* > + * The hardware should be setting the stop bit of CP_CONTEXT_SWITCH_CNTL > + * to zero before firing the interrupt, but there is a non zero chance > + * of a hardware condition or a software race that could set it again > + * before we have a chance to finish. If that happens, log and go for > + * recovery > + */ > + status = gpu_read(gpu, REG_A8XX_CP_CONTEXT_SWITCH_CNTL); > + if (unlikely(status & A8XX_CP_CONTEXT_SWITCH_CNTL_STOP)) { > + DRM_DEV_ERROR(&gpu->pdev->dev, > + "!!!!!!!!!!!!!!!! preemption faulted !!!!!!!!!!!!!! irq\n"); > + set_preempt_state(a6xx_gpu, PREEMPT_FAULTED); > + dev_err(dev->dev, "%s: Preemption failed to complete\n", > + gpu->name); > + kthread_queue_work(gpu->worker, &gpu->recover_work); > + return; > + } > + > + a6xx_gpu->cur_ring = a6xx_gpu->next_ring; > + a6xx_gpu->next_ring = NULL; > + > + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); > + > + update_wptr(a6xx_gpu, a6xx_gpu->cur_ring); > + > + set_preempt_state(a6xx_gpu, PREEMPT_NONE); > + > + a8xx_preempt_keepalive_vote(gpu, false); > + > + trace_msm_gpu_preemption_irq(a6xx_gpu->cur_ring->id); > + > + /* > + * Retrigger preemption to avoid a deadlock that might occur when preemption > + * is skipped due to it being already in flight when requested. > + */ > + a8xx_preempt_trigger(gpu); > +} > + > +void a8xx_preempt_hw_init(struct msm_gpu *gpu) > +{ > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + int i; > + > + /* No preemption if we only have one ring */ > + if (gpu->nr_rings == 1) > + return; > + > + for (i = 0; i < gpu->nr_rings; i++) { > + struct a6xx_preempt_record *record_ptr = a6xx_gpu->preempt[i]; > + > + record_ptr->wptr = 0; > + record_ptr->rptr = 0; > + record_ptr->rptr_addr = shadowptr(a6xx_gpu, gpu->rb[i]); > + record_ptr->info = 0; > + record_ptr->data = 0; > + record_ptr->rbase = gpu->rb[i]->iova; > + } > + > + /* Write a 0 to signal that we aren't switching pagetables */ > + gpu_write64(gpu, REG_A8XX_CP_CONTEXT_SWITCH_SMMU_INFO, 0); > + > + /* Enable the GMEM save/restore feature for preemption */ > + gpu_write(gpu, REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE_ENABLE, 0x1); > + > + /* Reset the preemption state */ > + set_preempt_state(a6xx_gpu, PREEMPT_NONE); > + > + spin_lock_init(&a6xx_gpu->eval_lock); > + > + /* Always come up on rb 0 */ > + a6xx_gpu->cur_ring = gpu->rb[0]; > +} > + > +void a8xx_preempt_trigger(struct msm_gpu *gpu) > +{ > + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); > + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); > + unsigned long flags; > + struct msm_ringbuffer *ring; > + unsigned int cntl; > + bool sysprof; > + > + if (gpu->nr_rings == 1) > + return; > + > + /* > + * Lock to make sure another thread attempting preemption doesn't skip it > + * while we are still evaluating the next ring. This makes sure the other > + * thread does start preemption if we abort it and avoids a soft lock. > + */ > + spin_lock_irqsave(&a6xx_gpu->eval_lock, flags); > + > + /* > + * Try to start preemption by moving from NONE to START. If > + * unsuccessful, a preemption is already in flight > + */ > + if (!try_preempt_state(a6xx_gpu, PREEMPT_NONE, PREEMPT_START)) { > + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); > + return; > + } > + > + cntl = A8XX_CP_CONTEXT_SWITCH_CNTL_LEVEL(a6xx_gpu->preempt_level); > + > + if (a6xx_gpu->skip_save_restore) > + cntl |= A8XX_CP_CONTEXT_SWITCH_CNTL_SKIP_SAVE_RESTORE; > + > + if (a6xx_gpu->uses_gmem) > + cntl |= A8XX_CP_CONTEXT_SWITCH_CNTL_USES_GMEM; > + > + cntl |= A8XX_CP_CONTEXT_SWITCH_CNTL_STOP; > + > + /* Get the next ring to preempt to */ > + ring = get_next_ring(gpu); > + > + /* > + * If no ring is populated or the highest priority ring is the current > + * one do nothing except to update the wptr to the latest and greatest > + */ > + if (!ring || (a6xx_gpu->cur_ring == ring)) { > + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); > + update_wptr(a6xx_gpu, a6xx_gpu->cur_ring); > + set_preempt_state(a6xx_gpu, PREEMPT_NONE); > + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); > + return; > + } > + > + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); > + > + spin_lock_irqsave(&ring->preempt_lock, flags); > + > + struct a7xx_cp_smmu_info *smmu_info_ptr = > + a6xx_gpu->preempt_smmu[ring->id]; > + struct a6xx_preempt_record *record_ptr = a6xx_gpu->preempt[ring->id]; > + u64 ttbr0 = ring->memptrs->ttbr0; > + u32 context_idr = ring->memptrs->context_idr; Shall we declare these variables at the top of the function body? Thanks, Jie > + > + smmu_info_ptr->ttbr0 = ttbr0; > + smmu_info_ptr->context_idr = context_idr; > + record_ptr->wptr = get_wptr(ring); > + > + /* > + * The GPU will write the wptr we set above when we preempt. Reset > + * restore_wptr to make sure that we don't write WPTR to the same > + * thing twice. It's still possible subsequent submissions will update > + * wptr again, in which case they will set the flag to true. This has > + * to be protected by the lock for setting the flag and updating wptr > + * to be atomic. > + */ > + ring->restore_wptr = false; > + > + trace_msm_gpu_preemption_trigger(a6xx_gpu->cur_ring->id, ring->id); > + > + spin_unlock_irqrestore(&ring->preempt_lock, flags); > + > + /* Set the keepalive bit to keep the GPU ON until preemption is complete */ > + a8xx_preempt_keepalive_vote(gpu, true); > + > + a6xx_fenced_write(a6xx_gpu, > + REG_A8XX_CP_CONTEXT_SWITCH_SMMU_INFO, a6xx_gpu->preempt_smmu_iova[ring->id], > + BIT(1), true); > + > + a6xx_fenced_write(a6xx_gpu, > + REG_A8XX_CP_CONTEXT_SWITCH_PRIV_NON_SECURE_RESTORE_ADDR, > + a6xx_gpu->preempt_iova[ring->id], BIT(1), true); > + > + a6xx_gpu->next_ring = ring; > + > + /* Start a timer to catch a stuck preemption */ > + mod_timer(&a6xx_gpu->preempt_timer, jiffies + msecs_to_jiffies(10000)); > + > + /* Enable or disable postamble as needed */ > + sysprof = refcount_read(&a6xx_gpu->base.base.sysprof_active) > 1; > + > + if (!sysprof && !a6xx_gpu->postamble_enabled) > + preempt_prepare_postamble(a6xx_gpu); > + > + if (sysprof && a6xx_gpu->postamble_enabled) > + preempt_disable_postamble(a6xx_gpu); > + > + /* Set the preemption state to triggered */ > + set_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED); > + > + /* Trigger the preemption */ > + a6xx_fenced_write(a6xx_gpu, REG_A8XX_CP_CONTEXT_SWITCH_CNTL, cntl, BIT(1), false); > +} > + >