* [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq
@ 2025-09-29 15:50 Marco Crivellari
2025-09-29 15:50 ` [PATCH v2 1/1] ASoC: replace use of " Marco Crivellari
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Marco Crivellari @ 2025-09-29 15:50 UTC (permalink / raw)
To: linux-kernel, linux-sound
Cc: Tejun Heo, Lai Jiangshan, Frederic Weisbecker,
Sebastian Andrzej Siewior, Marco Crivellari, Michal Hocko,
Liam Girdwood, Mark Brown, Jaroslav Kysela, Takashi Iwai
Hi!
Below is a summary of a discussion about the Workqueue API and cpu isolation
considerations. Details and more information are available here:
"workqueue: Always use wq_select_unbound_cpu() for WORK_CPU_UNBOUND."
https://lore.kernel.org/all/20250221112003.1dSuoGyc@linutronix.de/
=== Current situation: problems ===
Let's consider a nohz_full system with isolated CPUs: wq_unbound_cpumask is
set to the housekeeping CPUs, for !WQ_UNBOUND the local CPU is selected.
This leads to different scenarios if a work item is scheduled on an isolated
CPU where "delay" value is 0 or greater then 0:
schedule_delayed_work(, 0);
This will be handled by __queue_work() that will queue the work item on the
current local (isolated) CPU, while:
schedule_delayed_work(, 1);
Will move the timer on an housekeeping CPU, and schedule the work there.
Currently if a user enqueue a work item using schedule_delayed_work() the
used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use
WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to
schedule_work() that is using system_wq and queue_work(), that makes use
again of WORK_CPU_UNBOUND.
This lack of consistentcy cannot be addressed without refactoring the API.
=== Plan and future plans ===
This patchset is the first stone on a refactoring needed in order to
address the points aforementioned; it will have a positive impact also
on the cpu isolation, in the long term, moving away percpu workqueue in
favor to an unbound model.
These are the main steps:
1) API refactoring (that this patch is introducing)
- Make more clear and uniform the system wq names, both per-cpu and
unbound. This to avoid any possible confusion on what should be
used.
- Introduction of WQ_PERCPU: this flag is the complement of WQ_UNBOUND,
introduced in this patchset and used on all the callers that are not
currently using WQ_UNBOUND.
WQ_UNBOUND will be removed in a future release cycle.
Most users don't need to be per-cpu, because they don't have
locality requirements, because of that, a next future step will be
make "unbound" the default behavior.
2) Check who really needs to be per-cpu
- Remove the WQ_PERCPU flag when is not strictly required.
3) Add a new API (prefer local cpu)
- There are users that don't require a local execution, like mentioned
above; despite that, local execution yeld to performance gain.
This new API will prefer the local execution, without requiring it.
=== Introduced Changes by this series ===
1) [P 1] Replace use of system_wq
system_wq is a per-CPU worqueue, replaced by system_percpu_wq. Despite that,
system_wq in this change has been replaced by system_dfl_wq, because there
aren't per-cpu variables.
Thanks!
---
Changes in v2:
- Removed 1/2 from the series because already applied
- Instead of rename system_wq with system_percpu_wq, [P 1] change the workqueue
using system_dfl_wq (the new unbound workqueue).
Marco Crivellari (1):
ASoC: replace use of system_wq with system_dfl_wq
sound/soc/codecs/aw88081.c | 2 +-
sound/soc/codecs/aw88166.c | 2 +-
sound/soc/codecs/aw88261.c | 2 +-
sound/soc/codecs/aw88395/aw88395.c | 2 +-
sound/soc/codecs/aw88399.c | 2 +-
sound/soc/codecs/cs42l43-jack.c | 2 +-
sound/soc/codecs/cs42l43.c | 4 ++--
sound/soc/codecs/es8326.c | 12 ++++++------
sound/soc/codecs/rt5663.c | 6 +++---
sound/soc/intel/boards/sof_es8336.c | 2 +-
sound/soc/sof/intel/cnl.c | 2 +-
sound/soc/sof/intel/hda-ipc.c | 2 +-
12 files changed, 20 insertions(+), 20 deletions(-)
--
2.51.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v2 1/1] ASoC: replace use of system_wq with system_dfl_wq 2025-09-29 15:50 [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq Marco Crivellari @ 2025-09-29 15:50 ` Marco Crivellari 2025-09-29 16:58 ` [PATCH v2 0/1] ASoC: change " Mark Brown 2025-10-15 15:22 ` Mark Brown 2 siblings, 0 replies; 6+ messages in thread From: Marco Crivellari @ 2025-09-29 15:50 UTC (permalink / raw) To: linux-kernel, linux-sound Cc: Tejun Heo, Lai Jiangshan, Frederic Weisbecker, Sebastian Andrzej Siewior, Marco Crivellari, Michal Hocko, Liam Girdwood, Mark Brown, Jaroslav Kysela, Takashi Iwai Currently if a user enqueue a work item using schedule_delayed_work() the used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to schedule_work() that is using system_wq and queue_work(), that makes use again of WORK_CPU_UNBOUND. This lack of consistentcy cannot be addressed without refactoring the API. system_wq is a per-CPU worqueue, replaced by system_percpu_wq. Despite that, system_wq in this change has been replaced by system_dfl_wq, because there aren't per-cpu variables. The old wq will be kept for a few release cylces. Suggested-by: Tejun Heo <tj@kernel.org> Signed-off-by: Marco Crivellari <marco.crivellari@suse.com> --- sound/soc/codecs/aw88081.c | 2 +- sound/soc/codecs/aw88166.c | 2 +- sound/soc/codecs/aw88261.c | 2 +- sound/soc/codecs/aw88395/aw88395.c | 2 +- sound/soc/codecs/aw88399.c | 2 +- sound/soc/codecs/cs42l43-jack.c | 2 +- sound/soc/codecs/cs42l43.c | 4 ++-- sound/soc/codecs/es8326.c | 12 ++++++------ sound/soc/codecs/rt5663.c | 6 +++--- sound/soc/intel/boards/sof_es8336.c | 2 +- sound/soc/sof/intel/cnl.c | 2 +- sound/soc/sof/intel/hda-ipc.c | 2 +- 12 files changed, 20 insertions(+), 20 deletions(-) diff --git a/sound/soc/codecs/aw88081.c b/sound/soc/codecs/aw88081.c index d61a7b8c5470..34316748bbd1 100644 --- a/sound/soc/codecs/aw88081.c +++ b/sound/soc/codecs/aw88081.c @@ -779,7 +779,7 @@ static void aw88081_start(struct aw88081 *aw88081, bool sync_start) if (sync_start == AW88081_SYNC_START) aw88081_start_pa(aw88081); else - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &aw88081->start_work, AW88081_START_WORK_DELAY_MS); } diff --git a/sound/soc/codecs/aw88166.c b/sound/soc/codecs/aw88166.c index 28f62b991ef2..86b9d0709e50 100644 --- a/sound/soc/codecs/aw88166.c +++ b/sound/soc/codecs/aw88166.c @@ -1310,7 +1310,7 @@ static void aw88166_start(struct aw88166 *aw88166, bool sync_start) if (sync_start == AW88166_SYNC_START) aw88166_start_pa(aw88166); else - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &aw88166->start_work, AW88166_START_WORK_DELAY_MS); } diff --git a/sound/soc/codecs/aw88261.c b/sound/soc/codecs/aw88261.c index de11ae8dd9d9..ece6b4411c33 100644 --- a/sound/soc/codecs/aw88261.c +++ b/sound/soc/codecs/aw88261.c @@ -705,7 +705,7 @@ static void aw88261_start(struct aw88261 *aw88261, bool sync_start) if (sync_start == AW88261_SYNC_START) aw88261_start_pa(aw88261); else - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &aw88261->start_work, AW88261_START_WORK_DELAY_MS); } diff --git a/sound/soc/codecs/aw88395/aw88395.c b/sound/soc/codecs/aw88395/aw88395.c index fb563b4c6971..727aae2dd727 100644 --- a/sound/soc/codecs/aw88395/aw88395.c +++ b/sound/soc/codecs/aw88395/aw88395.c @@ -75,7 +75,7 @@ static void aw88395_start(struct aw88395 *aw88395, bool sync_start) if (sync_start == AW88395_SYNC_START) aw88395_start_pa(aw88395); else - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &aw88395->start_work, AW88395_START_WORK_DELAY_MS); } diff --git a/sound/soc/codecs/aw88399.c b/sound/soc/codecs/aw88399.c index 58846feb013d..6a05dbe5ee71 100644 --- a/sound/soc/codecs/aw88399.c +++ b/sound/soc/codecs/aw88399.c @@ -1340,7 +1340,7 @@ static void aw88399_start(struct aw88399 *aw88399, bool sync_start) if (sync_start == AW88399_SYNC_START) aw88399_start_pa(aw88399); else - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &aw88399->start_work, AW88399_START_WORK_DELAY_MS); } diff --git a/sound/soc/codecs/cs42l43-jack.c b/sound/soc/codecs/cs42l43-jack.c index 867e23d4fb8d..68a4ce7c2732 100644 --- a/sound/soc/codecs/cs42l43-jack.c +++ b/sound/soc/codecs/cs42l43-jack.c @@ -300,7 +300,7 @@ irqreturn_t cs42l43_bias_detect_clamp(int irq, void *data) { struct cs42l43_codec *priv = data; - queue_delayed_work(system_wq, &priv->bias_sense_timeout, + queue_delayed_work(system_dfl_wq, &priv->bias_sense_timeout, msecs_to_jiffies(1000)); return IRQ_HANDLED; diff --git a/sound/soc/codecs/cs42l43.c b/sound/soc/codecs/cs42l43.c index b61df09f20cf..a11d8ed8fa4e 100644 --- a/sound/soc/codecs/cs42l43.c +++ b/sound/soc/codecs/cs42l43.c @@ -161,7 +161,7 @@ static void cs42l43_hp_ilimit_clear_work(struct work_struct *work) priv->hp_ilimit_count--; if (priv->hp_ilimit_count) - queue_delayed_work(system_wq, &priv->hp_ilimit_clear_work, + queue_delayed_work(system_dfl_wq, &priv->hp_ilimit_clear_work, msecs_to_jiffies(CS42L43_HP_ILIMIT_DECAY_MS)); snd_soc_dapm_mutex_unlock(dapm); @@ -179,7 +179,7 @@ static irqreturn_t cs42l43_hp_ilimit(int irq, void *data) if (priv->hp_ilimit_count < CS42L43_HP_ILIMIT_MAX_COUNT) { if (!priv->hp_ilimit_count) - queue_delayed_work(system_wq, &priv->hp_ilimit_clear_work, + queue_delayed_work(system_dfl_wq, &priv->hp_ilimit_clear_work, msecs_to_jiffies(CS42L43_HP_ILIMIT_DECAY_MS)); priv->hp_ilimit_count++; diff --git a/sound/soc/codecs/es8326.c b/sound/soc/codecs/es8326.c index 78c4e68f6002..f3b25220141b 100644 --- a/sound/soc/codecs/es8326.c +++ b/sound/soc/codecs/es8326.c @@ -812,12 +812,12 @@ static void es8326_jack_button_handler(struct work_struct *work) press_count = 0; } button_to_report = cur_button; - queue_delayed_work(system_wq, &es8326->button_press_work, + queue_delayed_work(system_dfl_wq, &es8326->button_press_work, msecs_to_jiffies(35)); } else if (prev_button != cur_button) { /* mismatch, detect again */ prev_button = cur_button; - queue_delayed_work(system_wq, &es8326->button_press_work, + queue_delayed_work(system_dfl_wq, &es8326->button_press_work, msecs_to_jiffies(35)); } else { /* released or no pressed */ @@ -912,7 +912,7 @@ static void es8326_jack_detect_handler(struct work_struct *work) (ES8326_INT_SRC_PIN9 | ES8326_INT_SRC_BUTTON)); regmap_write(es8326->regmap, ES8326_SYS_BIAS, 0x1f); regmap_update_bits(es8326->regmap, ES8326_HP_DRIVER_REF, 0x0f, 0x0d); - queue_delayed_work(system_wq, &es8326->jack_detect_work, + queue_delayed_work(system_dfl_wq, &es8326->jack_detect_work, msecs_to_jiffies(400)); es8326->hp = 1; goto exit; @@ -923,7 +923,7 @@ static void es8326_jack_detect_handler(struct work_struct *work) regmap_write(es8326->regmap, ES8326_INT_SOURCE, (ES8326_INT_SRC_PIN9 | ES8326_INT_SRC_BUTTON)); es8326_enable_micbias(es8326->component); - queue_delayed_work(system_wq, &es8326->button_press_work, 10); + queue_delayed_work(system_dfl_wq, &es8326->button_press_work, 10); goto exit; } if ((iface & ES8326_HPBUTTON_FLAG) == 0x01) { @@ -958,10 +958,10 @@ static irqreturn_t es8326_irq(int irq, void *dev_id) goto out; if (es8326->jack->status & SND_JACK_HEADSET) - queue_delayed_work(system_wq, &es8326->jack_detect_work, + queue_delayed_work(system_dfl_wq, &es8326->jack_detect_work, msecs_to_jiffies(10)); else - queue_delayed_work(system_wq, &es8326->jack_detect_work, + queue_delayed_work(system_dfl_wq, &es8326->jack_detect_work, msecs_to_jiffies(300)); out: diff --git a/sound/soc/codecs/rt5663.c b/sound/soc/codecs/rt5663.c index 45057562c0c8..50fdeb8f0606 100644 --- a/sound/soc/codecs/rt5663.c +++ b/sound/soc/codecs/rt5663.c @@ -1859,7 +1859,7 @@ static irqreturn_t rt5663_irq(int irq, void *data) dev_dbg(regmap_get_device(rt5663->regmap), "%s IRQ queue work\n", __func__); - queue_delayed_work(system_wq, &rt5663->jack_detect_work, + queue_delayed_work(system_dfl_wq, &rt5663->jack_detect_work, msecs_to_jiffies(250)); return IRQ_HANDLED; @@ -1974,7 +1974,7 @@ static void rt5663_jack_detect_work(struct work_struct *work) cancel_delayed_work_sync( &rt5663->jd_unplug_work); } else { - queue_delayed_work(system_wq, + queue_delayed_work(system_dfl_wq, &rt5663->jd_unplug_work, msecs_to_jiffies(500)); } @@ -2024,7 +2024,7 @@ static void rt5663_jd_unplug_work(struct work_struct *work) SND_JACK_BTN_0 | SND_JACK_BTN_1 | SND_JACK_BTN_2 | SND_JACK_BTN_3); } else { - queue_delayed_work(system_wq, &rt5663->jd_unplug_work, + queue_delayed_work(system_dfl_wq, &rt5663->jd_unplug_work, msecs_to_jiffies(500)); } } diff --git a/sound/soc/intel/boards/sof_es8336.c b/sound/soc/intel/boards/sof_es8336.c index 10b189ea88db..41b7545581e0 100644 --- a/sound/soc/intel/boards/sof_es8336.c +++ b/sound/soc/intel/boards/sof_es8336.c @@ -163,7 +163,7 @@ static int sof_es8316_speaker_power_event(struct snd_soc_dapm_widget *w, priv->speaker_en = !SND_SOC_DAPM_EVENT_ON(event); - queue_delayed_work(system_wq, &priv->pcm_pop_work, msecs_to_jiffies(70)); + queue_delayed_work(system_dfl_wq, &priv->pcm_pop_work, msecs_to_jiffies(70)); return 0; } diff --git a/sound/soc/sof/intel/cnl.c b/sound/soc/sof/intel/cnl.c index 385e5339f0a4..c4b58738d74b 100644 --- a/sound/soc/sof/intel/cnl.c +++ b/sound/soc/sof/intel/cnl.c @@ -329,7 +329,7 @@ int cnl_ipc_send_msg(struct snd_sof_dev *sdev, struct snd_sof_ipc_msg *msg) * CTX_SAVE IPC, which is sent before the DSP enters D3. */ if (hdr->cmd != (SOF_IPC_GLB_PM_MSG | SOF_IPC_PM_CTX_SAVE)) - mod_delayed_work(system_wq, &hdev->d0i3_work, + mod_delayed_work(system_dfl_wq, &hdev->d0i3_work, msecs_to_jiffies(SOF_HDA_D0I3_WORK_DELAY_MS)); return 0; diff --git a/sound/soc/sof/intel/hda-ipc.c b/sound/soc/sof/intel/hda-ipc.c index f3fbf43a70c2..94425c510861 100644 --- a/sound/soc/sof/intel/hda-ipc.c +++ b/sound/soc/sof/intel/hda-ipc.c @@ -96,7 +96,7 @@ void hda_dsp_ipc4_schedule_d0i3_work(struct sof_intel_hda_dev *hdev, if (hda_dsp_ipc4_pm_msg(msg_data->primary)) return; - mod_delayed_work(system_wq, &hdev->d0i3_work, + mod_delayed_work(system_dfl_wq, &hdev->d0i3_work, msecs_to_jiffies(SOF_HDA_D0I3_WORK_DELAY_MS)); } EXPORT_SYMBOL_NS(hda_dsp_ipc4_schedule_d0i3_work, "SND_SOC_SOF_INTEL_HDA_COMMON"); -- 2.51.0 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq 2025-09-29 15:50 [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq Marco Crivellari 2025-09-29 15:50 ` [PATCH v2 1/1] ASoC: replace use of " Marco Crivellari @ 2025-09-29 16:58 ` Mark Brown 2025-09-30 7:08 ` Marco Crivellari 2025-10-15 15:22 ` Mark Brown 2 siblings, 1 reply; 6+ messages in thread From: Mark Brown @ 2025-09-29 16:58 UTC (permalink / raw) To: Marco Crivellari Cc: linux-kernel, linux-sound, Tejun Heo, Lai Jiangshan, Frederic Weisbecker, Sebastian Andrzej Siewior, Michal Hocko, Liam Girdwood, Jaroslav Kysela, Takashi Iwai [-- Attachment #1: Type: text/plain, Size: 532 bytes --] On Mon, Sep 29, 2025 at 05:50:52PM +0200, Marco Crivellari wrote: > Hi! > > Below is a summary of a discussion about the Workqueue API and cpu isolation > considerations. Details and more information are available here: Please don't send cover letters for single patches, if there is anything that needs saying put it in the changelog of the patch or after the --- if it's administrative stuff. This reduces mail volume and ensures that any important information is recorded in the changelog rather than being lost. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq 2025-09-29 16:58 ` [PATCH v2 0/1] ASoC: change " Mark Brown @ 2025-09-30 7:08 ` Marco Crivellari 0 siblings, 0 replies; 6+ messages in thread From: Marco Crivellari @ 2025-09-30 7:08 UTC (permalink / raw) To: Mark Brown Cc: linux-kernel, linux-sound, Tejun Heo, Lai Jiangshan, Frederic Weisbecker, Sebastian Andrzej Siewior, Michal Hocko, Liam Girdwood, Jaroslav Kysela, Takashi Iwai On Mon, Sep 29, 2025 at 6:58 PM Mark Brown <broonie@kernel.org> wrote: > Please don't send cover letters for single patches, if there is anything > that needs saying put it in the changelog of the patch or after the --- > if it's administrative stuff. This reduces mail volume and ensures that > any important information is recorded in the changelog rather than being > lost. Ok Mark, thank you. The changelog already contains all the information. Noted for the next time. -- Marco Crivellari L3 Support Engineer, Technology & Product ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq 2025-09-29 15:50 [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq Marco Crivellari 2025-09-29 15:50 ` [PATCH v2 1/1] ASoC: replace use of " Marco Crivellari 2025-09-29 16:58 ` [PATCH v2 0/1] ASoC: change " Mark Brown @ 2025-10-15 15:22 ` Mark Brown 2025-10-15 15:23 ` Marco Crivellari 2 siblings, 1 reply; 6+ messages in thread From: Mark Brown @ 2025-10-15 15:22 UTC (permalink / raw) To: linux-kernel, linux-sound, Marco Crivellari Cc: Tejun Heo, Lai Jiangshan, Frederic Weisbecker, Sebastian Andrzej Siewior, Michal Hocko, Liam Girdwood, Jaroslav Kysela, Takashi Iwai On Mon, 29 Sep 2025 17:50:52 +0200, Marco Crivellari wrote: > Below is a summary of a discussion about the Workqueue API and cpu isolation > considerations. Details and more information are available here: > > "workqueue: Always use wq_select_unbound_cpu() for WORK_CPU_UNBOUND." > https://lore.kernel.org/all/20250221112003.1dSuoGyc@linutronix.de/ > > === Current situation: problems === > > [...] Applied to https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-next Thanks! [1/1] ASoC: replace use of system_wq with system_dfl_wq commit: 0b0eb7702a9fa410755e86124b4b7cd36e7d1cb4 All being well this means that it will be integrated into the linux-next tree (usually sometime in the next 24 hours) and sent to Linus during the next merge window (or sooner if it is a bug fix), however if problems are discovered then the patch may be dropped or reverted. You may get further e-mails resulting from automated or manual testing and review of the tree, please engage with people reporting problems and send followup patches addressing any issues that are reported if needed. If any updates are required or you are submitting further changes they should be sent as incremental updates against current git, existing patches will not be replaced. Please add any relevant lists and maintainers to the CCs when replying to this mail. Thanks, Mark ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq 2025-10-15 15:22 ` Mark Brown @ 2025-10-15 15:23 ` Marco Crivellari 0 siblings, 0 replies; 6+ messages in thread From: Marco Crivellari @ 2025-10-15 15:23 UTC (permalink / raw) To: Mark Brown Cc: linux-kernel, linux-sound, Tejun Heo, Lai Jiangshan, Frederic Weisbecker, Sebastian Andrzej Siewior, Michal Hocko, Liam Girdwood, Jaroslav Kysela, Takashi Iwai On Wed, Oct 15, 2025 at 5:22 PM Mark Brown <broonie@kernel.org> wrote: > [..] > Applied to > > https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-next > > Thanks! > > [1/1] ASoC: replace use of system_wq with system_dfl_wq > commit: 0b0eb7702a9fa410755e86124b4b7cd36e7d1cb4 > Many thanks, Mark! -- Marco Crivellari L3 Support Engineer, Technology & Product ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-10-15 15:24 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-09-29 15:50 [PATCH v2 0/1] ASoC: change system_wq with system_dfl_wq Marco Crivellari 2025-09-29 15:50 ` [PATCH v2 1/1] ASoC: replace use of " Marco Crivellari 2025-09-29 16:58 ` [PATCH v2 0/1] ASoC: change " Mark Brown 2025-09-30 7:08 ` Marco Crivellari 2025-10-15 15:22 ` Mark Brown 2025-10-15 15:23 ` Marco Crivellari
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®