From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752091AbdBOU6O (ORCPT ); Wed, 15 Feb 2017 15:58:14 -0500 Received: from mail-sn1nam01on0084.outbound.protection.outlook.com ([104.47.32.84]:13151 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752382AbdBOU5s (ORCPT ); Wed, 15 Feb 2017 15:57:48 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Suravee.Suthikulpanit@amd.com; From: Suravee Suthikulpanit To: , CC: , , , , Suravee Suthikulpanit , Suravee Suthikulpanit Subject: [PATCH v10 9/9] perf/amd/iommu: Enable support for multiple IOMMUs Date: Wed, 15 Feb 2017 14:56:25 -0600 Message-ID: <1487192185-31272-10-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1487192185-31272-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1487192185-31272-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [114.109.128.54] X-ClientProxiedBy: SG2PR01CA0102.apcprd01.prod.exchangelabs.com (10.170.138.156) To BN6PR12MB1444.namprd12.prod.outlook.com (10.172.24.21) X-MS-Office365-Filtering-Correlation-Id: 7e5e2579-2a78-4472-ae56-08d455e54a58 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081);SRVR:BN6PR12MB1444; X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1444;3:Q7dcodEBOasTelAM4r5FtHpGSfe4q6fq/8vqjtapi5jCX0sAlcWNxKIoANERVeZUff5BaZRsb78s7g1rXn5CT6s3+thKx3uLYFgVZUYqR/z3kwSAdwXLEndY6XgqqEAPQtc6aQO/Mj/H77ekIN1ddNtXXDet0ZJ3AfG6qGWFinFb3rgvkahjT+4BSwGN3RjZsprEfhiHFHmElvHfgGz2khf4jiSF4rL5tsg5GDrkOZC6uHBfOnRUuQbN3TBDUDamsqcTjlxKJZE4pIS4+9e9J6vJs6BuMsgw7N2q8g/YN6g=;25:aLwIiY7+Qq5LtbaBnsrcIPetWk3dpGiB5ey/B4/CFCM/lb3kMTCJ3AosA3lvFHR5wJYTg+oDliTvBSHCd9baDhnLQ41gLwHNGJZ5QXDjjJ2DL8FGLJGUIHUyUT4UTjO0+RrfY/NPrK2jtoW3Y76VRfxoG+LpokC8qJq81YQaCGsfbMdclP/ABSSseTXuWXfnaD6fRas/99sxRzlaF946a14w5SheVmlstm14V6/TC46BnLE9qL9apLZtZ9X42ReqOqjgnPo4xh9blnjDOwLGBrJGWSLKDBGRd0BFwNjuCm0qwixQELcoVXHplVf8nk36Ux1xUqK01Elu+QdtuYIP8Ix5AtIcGCijBlO3zDNF/jk+ChS5/FDPKV1yIJQpmhapxrb/rLFKEe9QepQwvmbLh47Zusdo1arWYgVx3O8keWlDcDWX+4r+jCO4HfcOGKUGOiX9hnoNKh7jfEvX3FMoTg== X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1444;31:/iaB72yq8IADLDxD35yBHvF43kx5z5kOFvR6ZiAYZd4MjEeN+YU7cSSE2yNDW2LQPYk+LDU3OOL2bQuCH4YnRfT14SQtnonevP2viRuCf3qBml0R7f/cAGywCMgn5rPd8J8+cWS7v9Tf1GDXEDSiEVksCCN5SshjDy6GSRCkSbv5AT+pi9SdPAnE2SDXMFuPDdQccDfLCSaLgxE7xRGFkM4OA7bSJtcvge28JB+/lFk=;20:wBhzHafgH4UUyK4ZXRWoB9IYlIxrU9rabQ2BExy+DcANJuQH9X+9TjoSXZPuf0AXW3rMz4ptI3YAjtIEAR82ntvVLBJhYjsLn5W1kqC6EV0NgVz+0YIWqiaZykTYxqzebNz7OCLDyH29vt+5nKNoQVcQEqGUDa7ef+0JJU1UmLj3UOpBRlkwrdoOKUbHNkkQ5BltAi7ck5Sni73rOf07tuWp6dehOtzNyrIwMYLhWgWo/vlZ94KpOxBht36mfarOjssD33tdA5q9AkStjgtH4L6bnAePdcA9O7AV06OX5tLYgnwwIyeAp4xGcDsKl1hMZopMH+okEPhiSgwlEDbaFQiVWrIfAKMShDgezbyGiX3nKY9M/pnbfwEp9EVwZ0vTBzOqAY9EGAamTvwwMnxPcCzeR6Q1shagRCoYQzSTK0dXRnnhQYuEIcl5aSdnNgLUzgrLisRv1BHtaILaIhhKBpRMJVS0UGCJ+fQzKSFHuQegQseHW+bsfk8jv4hV61BU X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6041248)(20161123564025)(20161123562025)(20161123558025)(20161123560025)(20161123555025)(6072148);SRVR:BN6PR12MB1444;BCL:0;PCL:0;RULEID:;SRVR:BN6PR12MB1444; X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1444;4:mEN2V5ZSGu7ycP6jtL9KMu4WWDwzMqVxlyjgRUGfRmreivaHZuVQpXjap1kifOid2LvB+TDS7lLOksOiq7UyJFY/QXyQZKQ2yZvnDVk5vuKzM6D6gYtCCW8X0Aco6F4PoXgR4NoAsl14WjlHLJpSJ8n+7YQWq0gX5rYpX/u5BWtEtF4F9HdjEjJRM60sma1TdSfrYhL1xF+fkugDPt55gErQ63goSfXxrG7aPiIDEqRqvFzXg9zhzZ7yQIiqBXPAVC5kOdYpokLwVGuL8CrPA9vPluW2EThsjnt50+2j1jCCZ5fMTKnXGOTHq+BKaBhqOx6GkaurC2Jt6n1u4BzW1H5/J19vMSpuC4CreRPl5BRqYsb3609He/3bcDGq4iLMVku+Pn0dHJAY80J+mQK5MP4R5lHaTYsa6iYseNgbG7jLl9OxckMwYdSGQ/Yxz1GwFgAAupuSARp971JwWw2BNpiWEWnzEnz9CrGP7fpRqh1Yz4NLnQnUZoQpEaBmn6JLKsHlewMN0mI0DJMgunPrCYCT5Me9YVuTDo01gmVZHNis6+nyZsiaCjExHApjthekvh/NDVNGR8mYBEPhivngLi/tVxzTYuVbuwm1sWej65v5Yad+kDDjcpqvDNrAFtiv4A2MjLlyAUQkoAJTdR4qsXP5gho5F7Eociy1Af+DSfI= X-Forefront-PRVS: 021975AE46 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(39840400002)(39450400003)(39860400002)(39850400002)(39410400002)(189002)(199003)(6486002)(50986999)(54906002)(38730400002)(53936002)(86362001)(305945005)(48376002)(25786008)(5003940100001)(7736002)(389900002)(42186005)(106356001)(2950100002)(4720700003)(5660300001)(76176999)(105586002)(6666003)(6512007)(189998001)(68736007)(47776003)(50226002)(97736004)(101416001)(50466002)(81166006)(66066001)(81156014)(8676002)(36756003)(6506006)(2906002)(3846002)(4326007)(92566002)(6116002);DIR:OUT;SFP:1101;SCL:1;SRVR:BN6PR12MB1444;H:ssuthiku-rhel73.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BN6PR12MB1444;23:PQDzCB6V03L+b4HwmvNegRaJkVKkTwx9UiRhwyj2J?= =?us-ascii?Q?kjHpx9xhVPO/opjQ702KlCzsYzErQL3CBJKCoWsSK8AVwN2t9Qdt5hT1wHL5?= =?us-ascii?Q?E6tIP6yBlDjW56qMqHgAAztGZgfJa8rQ5xPaUm07r8Zrq3r2DVRXpgbDD1CA?= =?us-ascii?Q?K9pK+EHcUEulPY6VBZlGbC+3T8wFD8nQm4OThKQadKhppU7TaqelGw6pzXBw?= =?us-ascii?Q?SlIhrjPEgYCYEODSu+VOVHgolCleWkfQPbBJ1ITXNhbHfu5iRJsnKKdaJln5?= =?us-ascii?Q?yLBRm6pcxrljHqbLAtWOSR60J2JcR+JMUUJcrURIPBAJqWDPlrXIh/KRy4co?= =?us-ascii?Q?lQ4Cn7aggMB/q2W2lkJ2hQPIMxPJkd/IVxhuHqCqdcoo70B+eUhoeYxaYzt5?= =?us-ascii?Q?8qRWR2IgcqvkhS5GlB82m2767lFQFKWvHtMZ3mu/6cGu7e+l1qo7R9NPBbNZ?= =?us-ascii?Q?Zb2uuVUZbCrtyDkxCt66tDEnQQ+9Em4iX1F3YaNrGt7S33ebeitRrzSiHAFb?= =?us-ascii?Q?LKKBCVdmMSl55zf3j3SzKtSBOsuIuDKYM4QHxxNByTDErFUFaUJgYSxY9IzP?= =?us-ascii?Q?x8GYI3Znx7Qxq8Ya/0QcOYvrVtoI867tZCtw1YWBsMtp0Z/LvHKVArseOTR6?= =?us-ascii?Q?8EcVcy3jzoW4mtit7/6IAdmv9CB4cB9uZP1aSZvGjXnnchma2FFs41uMxOxC?= =?us-ascii?Q?jfBJzm6pmuFi3BUROJA1haeYJj9KUpW7XH54F8FCrJpHghJluvm3FOpkpbDK?= =?us-ascii?Q?GjRuvqAj6BkDYBOEOEfI39mnRcen2o9IAhv1Q45wmRoCY89P1RutqDuQi4lM?= =?us-ascii?Q?8NP0A6eRm7+qIS4NejmVAjOI0zUqIH2HR3W5J+jmAEEVgVkWzOhLko1DPgTh?= =?us-ascii?Q?eqri2w/QsGU9iTuHbC2Vymrm4CxdsmZ9Ub9ow0YpzAUSIDr69EyfdiioFjhh?= =?us-ascii?Q?4rhNZJrfbUoi7WBN8ikmBh4GAMlb8EQ0PsEW+1SaVa3uUDPXaKqXZNnYIRLy?= =?us-ascii?Q?ik9UqK28xQaFvlGERO5xh3BxczS5sQbNXzqOMe1zHSNuQ+wBLy+gh7iIBljL?= =?us-ascii?Q?i7QH1+wA+ls19c1uZ1iRY/wdIWHEuFPlagIysnqZxBlt/BoB81Qa07EtFkbv?= =?us-ascii?Q?BHGW5OPm6quh4SUrgPek3FHfR48X6KUnW5Rnwi8dsThCn8bbdgDegWnkZv6j?= =?us-ascii?Q?aBLNJoqIpgP0s5h38cvsVgZUk4uaGsO5l5i?= X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1444;6:QPF9+JQ+9mMinFrzVAnHJlF/65ZusalRQ5kracn/Tahre6i0OgY4dwhpHgafApABBUfQP0FAyJGQ6ly5Zmy0Z0xDygIxL9JzDHhTsE4CfuFI7sWUVEoxG6i3N8jVk56XH4l5vhmt1rrOUAz6qFKDlwu+s0fMH7Dg2DFm1cIPtWgSYvN6DOsfffoSriesSMt0v88QryIyKR95B/lKiONiNX+8xMjd8OLO1P47v8BykgCOLeWBNaNXv7eNmYo0A1jvZjwDIGlM0vrkU5jpIwG2rLMH0gLPTu9TH5uGR6KeBuFlxLkwgpmDSpM9rxtmTIJOHVeYBKDY4PuPEdCpeKMQ+19dLkcCiUNU/I1L5NJhkciY1uNDVjZK0mJV0qqHjqFsQVKTSxa+yvw9k6wlKbBbBB90nynmgJ8fOGpoRL7+I5I=;5:sFLh8iaSeUPRaE9/mRwgBQsbTgUqqLHYQYLTvQ6Ywf1VOXAARrYKFQEItwy52MoMxykfU8ejPBmkyxS5Xaat8oG2MTV9/E8Ajtkb4M+M9rMj/W05/edsJvy3lyACkA6qtAco6XVO3RxJrjzQrdRNQQ==;24:ABdE4cQdJaS0S56gvtwl6UOFa+DaOkBfEaL2OdfvUo8KBeIZtp+ZOgKzjwsCyaGOa1LomWH8iGcc0+TrQ8mz2UL8LpTLBRt4AWT9996WyOs= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1444;7:tDN4QRxbM7QQtOMQxMd9MErcBW+3Eug75IQFphDUMbNMFsq3x6AmR7LRL5EkXCiZy9r58thPsrQXk03CcVylnP2krNDobV3UbfLAmh9O0DFC71EyX1q1X4jgWPtic7Ue2pwxrDduR6N1/fOTeVnj8tMz4Uv+jPQI+kBUttnGoQiiVBRPRHyuvz7QLi314YJqXspad4yScN09TCDIPJWxp8LxV95cZDO5MMvKDR0X4okXuy+SgbX+ImGQaViQn5gNEYZYC2mNKsDVaItPmDYrH078dfYNyEbchMl1bDOQPARJMurQ5jl0ACt5NoaWWM1g+pJsE8i20c3TKzLtxfeC5g==;20:vEsxznQ7EvgIs25oZcPBT7McdnD+BCk/0J4wZJJWHvMJijXoCAA0KnJk3ldR9dqlMVXdWSl8vFJWmr9l4S8S51lUaDwqcZH1m/QPnV8MvlDvAxELAZfRGspUwXbGEg0PbwsF03pxM91Us7h8At2hqTXYZO1ju8yKJEOXsP+TBkrFBqhVx8gC4IQA+AhGTTN7LWzFCC9NrywEyn3IVAgmSc2W7q0U+c+vxECEZr0PVKWXl1hDKT6WSPDvqRvqJLwB X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2017 20:57:37.1175 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1444 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suravee Suthikulpanit Add multi-IOMMU support for perf by exposing an AMD IOMMU PMU for each IOMMU found in the system via: /bus/event_source/devices/amd_iommu_x where x is the IOMMU index. This allows users to specify different events to be programmed onto performance counters of each IOMMU. Cc: Borislav Petkov Cc: Peter Zijlstra Signed-off-by: Suravee Suthikulpanit --- arch/x86/events/amd/iommu.c | 129 ++++++++++++++++++++++++-------------------- 1 file changed, 72 insertions(+), 57 deletions(-) diff --git a/arch/x86/events/amd/iommu.c b/arch/x86/events/amd/iommu.c index 7bbf405..cc1d2eb 100644 --- a/arch/x86/events/amd/iommu.c +++ b/arch/x86/events/amd/iommu.c @@ -35,16 +35,21 @@ #define _GET_PASID_MASK(ev) ((ev->hw.extra_reg.config >> 16) & 0xFFFFULL) #define _GET_DOMID_MASK(ev) ((ev->hw.extra_reg.config >> 32) & 0xFFFFULL) -static struct perf_amd_iommu __perf_iommu; +#define PERF_AMD_IOMMU_NAME_SIZE 16 struct perf_amd_iommu { + struct list_head list; struct pmu pmu; + struct amd_iommu *iommu; + char name[PERF_AMD_IOMMU_NAME_SIZE]; u8 max_banks; u8 max_counters; u64 cntr_assign_mask; raw_spinlock_t lock; }; +static LIST_HEAD(perf_amd_iommu_list); + /*--------------------------------------------- * sysfs format attributes *---------------------------------------------*/ @@ -202,8 +207,6 @@ static int clear_avail_iommu_bnk_cntr(struct perf_amd_iommu *perf_iommu, static int perf_iommu_event_init(struct perf_event *event) { struct hw_perf_event *hwc = &event->hw; - struct perf_amd_iommu *perf_iommu; - u64 config, config1; /* test the event attr type check for PMU enumeration */ if (event->attr.type != event->pmu->type) @@ -225,28 +228,21 @@ static int perf_iommu_event_init(struct perf_event *event) if (event->cpu < 0) return -EINVAL; - perf_iommu = &__perf_iommu; - - if (event->pmu != &perf_iommu->pmu) - return -ENOENT; - - if (perf_iommu) { - config = event->attr.config; - config1 = event->attr.config1; - } else { - return -EINVAL; - } - /* update the hw_perf_event struct with the iommu config data */ - hwc->config = config; - hwc->extra_reg.config = config1; + hwc->config = event->attr.config; + hwc->extra_reg.config = event->attr.config1; return 0; } +static inline struct amd_iommu *perf_event_2_iommu(struct perf_event *ev) +{ + return (container_of(ev->pmu, struct perf_amd_iommu, pmu))->iommu; +} + static void perf_iommu_enable_event(struct perf_event *ev) { - struct amd_iommu *iommu = get_amd_iommu(0); + struct amd_iommu *iommu = perf_event_2_iommu(ev); u8 csource = _GET_CSOURCE(ev); u16 devid = _GET_DEVID(ev); u8 bank = _GET_BANK(ev); @@ -274,7 +270,7 @@ static void perf_iommu_enable_event(struct perf_event *ev) static void perf_iommu_disable_event(struct perf_event *event) { - struct amd_iommu *iommu = get_amd_iommu(0); + struct amd_iommu *iommu = perf_event_2_iommu(event); u64 reg = 0ULL; amd_iommu_pc_set_reg(iommu, _GET_BANK(event), _GET_CNTR(event), @@ -284,7 +280,7 @@ static void perf_iommu_disable_event(struct perf_event *event) static void perf_iommu_start(struct perf_event *event, int flags) { struct hw_perf_event *hwc = &event->hw; - struct amd_iommu *iommu = get_amd_iommu(0); + struct amd_iommu *iommu = perf_event_2_iommu(event); if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED))) return; @@ -307,7 +303,7 @@ static void perf_iommu_read(struct perf_event *event) { u64 count, prev, delta; struct hw_perf_event *hwc = &event->hw; - struct amd_iommu *iommu = get_amd_iommu(0); + struct amd_iommu *iommu = perf_event_2_iommu(event); if (amd_iommu_pc_get_reg(iommu, _GET_BANK(event), _GET_CNTR(event), IOMMU_PC_COUNTER_REG, &count)) @@ -401,11 +397,6 @@ static __init int _init_events_attrs(void) return 0; } -static __init void amd_iommu_pc_exit(void) -{ - kfree(amd_iommu_events_group.attrs); -} - const struct attribute_group *amd_iommu_attr_groups[] = { &amd_iommu_format_group, &amd_iommu_cpumask_group, @@ -413,47 +404,56 @@ static __init void amd_iommu_pc_exit(void) NULL, }; +static struct pmu iommu_pmu = { + .event_init = perf_iommu_event_init, + .add = perf_iommu_add, + .del = perf_iommu_del, + .start = perf_iommu_start, + .stop = perf_iommu_stop, + .read = perf_iommu_read, + .task_ctx_nr = perf_invalid_context, + .attr_groups = amd_iommu_attr_groups, +}; + static __init int -_init_perf_amd_iommu(struct perf_amd_iommu *perf_iommu, char *name) +init_one_iommu(unsigned int idx) { int ret; + struct perf_amd_iommu *perf_iommu; - raw_spin_lock_init(&perf_iommu->lock); + perf_iommu = kzalloc(sizeof(struct perf_amd_iommu), GFP_KERNEL); + if (!perf_iommu) + return -ENOMEM; - /* Init cpumask attributes to only core 0 */ - cpumask_set_cpu(0, &iommu_cpumask); + raw_spin_lock_init(&perf_iommu->lock); - perf_iommu->max_banks = amd_iommu_pc_get_max_banks(0); - perf_iommu->max_counters = amd_iommu_pc_get_max_counters(0); - if (!perf_iommu->max_banks || !perf_iommu->max_counters) + perf_iommu->pmu = iommu_pmu; + perf_iommu->iommu = get_amd_iommu(idx); + perf_iommu->max_banks = amd_iommu_pc_get_max_banks(idx); + perf_iommu->max_counters = amd_iommu_pc_get_max_counters(idx); + if (!perf_iommu->iommu || !perf_iommu->max_banks || + !perf_iommu->max_counters) { + kfree(perf_iommu); return -EINVAL; + } - perf_iommu->pmu.attr_groups = amd_iommu_attr_groups; - ret = perf_pmu_register(&perf_iommu->pmu, name, -1); - if (ret) - pr_err("Error initializing AMD IOMMU perf counters.\n"); - else - pr_info("perf: amd_iommu: Detected. (%d banks, %d counters/bank)\n", - amd_iommu_pc_get_max_banks(0), - amd_iommu_pc_get_max_counters(0)); + snprintf(perf_iommu->name, PERF_AMD_IOMMU_NAME_SIZE, "amd_iommu_%u", idx); + ret = perf_pmu_register(&perf_iommu->pmu, perf_iommu->name, -1); + if (!ret) { + pr_info("Detected AMD IOMMU #%d (%d banks, %d counters/bank)\n", + idx, perf_iommu->max_banks, perf_iommu->max_counters); + list_add_tail(&perf_iommu->list, &perf_amd_iommu_list); + } else { + pr_warn("Error initializing IOMMU %d.\n", idx); + kfree(perf_iommu); + } return ret; } -static struct perf_amd_iommu __perf_iommu = { - .pmu = { - .task_ctx_nr = perf_invalid_context, - .event_init = perf_iommu_event_init, - .add = perf_iommu_add, - .del = perf_iommu_del, - .start = perf_iommu_start, - .stop = perf_iommu_stop, - .read = perf_iommu_read, - }, -}; - static __init int amd_iommu_pc_init(void) { int ret; + unsigned int i, cnt = 0; /* Make sure the IOMMU PC resource is available */ if (!amd_iommu_pc_supported()) @@ -463,11 +463,26 @@ static __init int amd_iommu_pc_init(void) if (ret) return ret; - ret = _init_perf_amd_iommu(&__perf_iommu, "amd_iommu"); - if (ret) - amd_iommu_pc_exit(); + /* + * An IOMMU PMU is specific to each IOMMU, and can + * function independently. So, we go through all IOMMUs + * and ignore the one that fails initialization unless + * all IOMMU are failing. + */ + for (i = 0; i < amd_iommu_get_num_iommus(); i++) { + ret = init_one_iommu(i); + if (!ret) + cnt++; + } - return ret; + if (!cnt) { + kfree(amd_iommu_events_group.attrs); + return -ENODEV; + } + + /* Init cpumask attributes to only core 0 */ + cpumask_set_cpu(0, &iommu_cpumask); + return 0; } device_initcall(amd_iommu_pc_init); -- 1.8.3.1