mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Qian Cai <cai@lca.pw>, Will Deacon <will.deacon@arm.com>,
	Sasha Levin <sashal@kernel.org>
Subject: [PATCH AUTOSEL 4.20 061/117] arm64: kasan: Increase stack size for KASAN_EXTRA
Date: Tue,  8 Jan 2019 14:25:29 -0500	[thread overview]
Message-ID: <20190108192628.121270-61-sashal@kernel.org> (raw)
In-Reply-To: <20190108192628.121270-1-sashal@kernel.org>

From: Qian Cai <cai@lca.pw>

[ Upstream commit 6e8830674ea77f57d57a33cca09083b117a71f41 ]

If the kernel is configured with KASAN_EXTRA, the stack size is
increased significantly due to setting the GCC -fstack-reuse option to
"none" [1]. As a result, it can trigger a stack overrun quite often with
32k stack size compiled using GCC 8. For example, this reproducer

  https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/madvise/madvise06.c

can trigger a "corrupted stack end detected inside scheduler" very
reliably with CONFIG_SCHED_STACK_END_CHECK enabled. There are other
reports at:

  https://lore.kernel.org/lkml/1542144497.12945.29.camel@gmx.us/
  https://lore.kernel.org/lkml/721E7B42-2D55-4866-9C1A-3E8D64F33F9C@gmx.us/

There are just too many functions that could have a large stack with
KASAN_EXTRA due to large local variables that have been called over and
over again without being able to reuse the stacks. Some noticiable ones
are,

size
7536 shrink_inactive_list
7440 shrink_page_list
6560 fscache_stats_show
3920 jbd2_journal_commit_transaction
3216 try_to_unmap_one
3072 migrate_page_move_mapping
3584 migrate_misplaced_transhuge_page
3920 ip_vs_lblcr_schedule
4304 lpfc_nvme_info_show
3888 lpfc_debugfs_nvmestat_data.constprop

There are other 49 functions over 2k in size while compiling kernel with
"-Wframe-larger-than=" on this machine. Hence, it is too much work to
change Makefiles for each object to compile without
-fsanitize-address-use-after-scope individually.

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715#c23

Signed-off-by: Qian Cai <cai@lca.pw>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/arm64/include/asm/memory.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index f0a5c9531e8b..778af0b7f7fd 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -67,12 +67,17 @@
 /*
  * KASAN requires 1/8th of the kernel virtual address space for the shadow
  * region. KASAN can bloat the stack significantly, so double the (minimum)
- * stack size when KASAN is in use.
+ * stack size when KASAN is in use, and then double it again if KASAN_EXTRA is
+ * on.
  */
 #ifdef CONFIG_KASAN
 #define KASAN_SHADOW_SCALE_SHIFT 3
 #define KASAN_SHADOW_SIZE	(UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT))
+#ifdef CONFIG_KASAN_EXTRA
+#define KASAN_THREAD_SHIFT	2
+#else
 #define KASAN_THREAD_SHIFT	1
+#endif /* CONFIG_KASAN_EXTRA */
 #else
 #define KASAN_SHADOW_SIZE	(0)
 #define KASAN_THREAD_SHIFT	0
-- 
2.19.1


  parent reply	other threads:[~2019-01-08 19:28 UTC|newest]

Thread overview: 135+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-08 19:24 [PATCH AUTOSEL 4.20 001/117] netfilter: ipset: Allow matching on destination MAC address for mac and ipmac sets Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 002/117] gpio: pl061: Move irq_chip definition inside struct pl061 Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 003/117] ASoC: wm97xx: fix uninitialized regmap pointer problem Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 004/117] drm/amd/display: Guard against null stream_state in set_crc_source Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 005/117] drm/amdkfd: fix interrupt spin lock Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 006/117] qtnfmac: fix error handling in control path Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 007/117] ixgbe: allow IPsec Tx offload in VEPA mode Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 008/117] platform/x86: asus-wmi: Tell the EC the OS will handle the display off hotkey Sasha Levin
2019-01-09  0:49   ` João Paulo Rechi Vita
2019-01-23 13:55     ` Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 009/117] e1000e: allow non-monotonic SYSTIM readings Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 010/117] usb: typec: tcpm: Do not disconnect link for self powered devices Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 011/117] selftests/bpf: enable (uncomment) all tests in test_libbpf.sh Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 012/117] of: overlay: add missing of_node_put() after add new node to changeset Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 013/117] writeback: don't decrement wb->refcnt if !wb->bdi Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 014/117] serial: set suppress_bind_attrs flag only if builtin Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 015/117] bpf: Allow narrow loads with offset > 0 Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask Sasha Levin
2019-01-09  6:50   ` Amir Goldstein
2019-01-09 11:32     ` Jan Kara
2019-01-23 13:56       ` Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 017/117] ALSA: oxfw: add support for APOGEE duet FireWire Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 018/117] x86/mce: Fix -Wmissing-prototypes warnings Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 019/117] MIPS: SiByte: Enable swiotlb for SWARM, LittleSur and BigSur Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 020/117] crypto: ecc - regularize scalar for scalar multiplication Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 021/117] drm/scheduler: Fix bad job be re-processed in TDR Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 022/117] arm64: perf: set suppress_bind_attrs flag to true Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 023/117] MIPS: Loongson: Add Loongson-3A R2.1 basic support Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 024/117] x86/fault: Don't try to recover from an implicit supervisor access Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 025/117] USB: serial: ftdi_sio: use rounding when calculating baud rate divisors Sasha Levin
2019-01-09  8:24   ` Johan Hovold
2019-01-23 13:58     ` Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 026/117] drm/atomic-helper: Complete fake_commit->flip_done potentially earlier Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 027/117] clk: meson: meson8b: add support for more M/N values in sys_pll Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 028/117] clk: meson: meson8b: fix incorrect divider mapping in cpu_scale_table Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 029/117] samples: bpf: fix: error handling regarding kprobe_events Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 030/117] usb: gadget: udc: renesas_usb3: add a safety connection way for forced_b_device Sasha Levin
2019-01-08 19:24 ` [PATCH AUTOSEL 4.20 031/117] fpga: altera-cvp: fix probing for multiple FPGAs on the bus Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 032/117] selinux: always allow mounting submounts Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 033/117] arm64: Use a raw spinlock in __install_bp_hardening_cb() Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 034/117] drm/amdgpu: Correct get_crtc_scanoutpos behavior when vpos >= vtotal Sasha Levin
2019-01-09  9:20   ` Michel Dänzer
2019-01-23 14:03     ` Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 035/117] block: use rcu_work instead of call_rcu to avoid sleep in softirq Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 036/117] ASoC: pcm3168a: Don't disable pcm3168a when CONFIG_PM defined Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 037/117] scsi: qedi: Check for session online before getting iSCSI TLV data Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 038/117] net: ethernet: ave: Set initial wol state to disabled Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 039/117] drm/amdgpu: Reorder uvd ring init before uvd resume Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 040/117] rxe: IB_WR_REG_MR does not capture MR's iova field Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 041/117] efi/libstub: Disable some warnings for x86{,_64} Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 042/117] jffs2: Fix use of uninitialized delayed_work, lockdep breakage Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 043/117] kbuild: let fixdep directly write to .*.cmd files Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 044/117] clk: imx: make mux parent strings const Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 045/117] pstore/ram: Do not treat empty buffers as valid Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 046/117] media: uvcvideo: Refactor teardown of uvc on USB disconnect Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 047/117] powerpc/xmon: Fix invocation inside lock region Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 048/117] powerpc/pseries/cpuidle: Fix preempt warning Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 049/117] arm64: relocatable: fix inconsistencies in linker script and options Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 050/117] media: firewire: Fix app_info parameter type in avc_ca{,_app}_info Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 051/117] ASoC: use dma_ops of parent device for acp_audio_dma Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 052/117] coresight: tmc: Fix bad register address for CLAIM Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 053/117] IB/hfi1: Incorrect sizing of sge for PIO will OOPs Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 054/117] media: venus: core: Set dma maximum segment size Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 055/117] staging: erofs: fix use-after-free of on-stack `z_erofs_vle_unzip_io' Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 056/117] media: cedrus: don't initialize pointers with zero Sasha Levin
2019-01-09  8:48   ` Dan Carpenter
2019-01-23 14:04     ` Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 057/117] net: call sk_dst_reset when set SO_DONTROUTE Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 058/117] scsi: target: use consistent left-aligned ASCII INQUIRY data Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 059/117] scsi: target/core: Make sure that target_wait_for_sess_cmds() waits long enough Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 060/117] selftests: do not macro-expand failed assertion expressions Sasha Levin
2019-01-08 19:25 ` Sasha Levin [this message]
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 062/117] clk: imx6q: reset exclusive gates on init Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 063/117] arm64: Fix minor issues with the dcache_by_line_op macro Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 064/117] bpf: relax verifier restriction on BPF_MOV | BPF_ALU Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 065/117] usb: dwc2: Fix disable all EP's on disconnect Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 066/117] x86/resctrl: Fix rdt_find_domain() return value and checks Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 067/117] kconfig: fix file name and line number of warn_ignored_character() Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 068/117] kconfig: fix memory leak when EOF is encountered in quotation Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 069/117] iio: adc: qcom-spmi-adc5: Initialize prescale properly Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 070/117] mmc: atmel-mci: do not assume idle after atmci_request_end Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 071/117] btrfs: volumes: Make sure there is no overlap of dev extents at mount time Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 072/117] btrfs: alloc_chunk: fix more DUP stripe size handling Sasha Levin
2019-01-08 23:52   ` Hans van Kranenburg
2019-01-23 14:37     ` Sasha Levin
2019-01-23 15:54       ` Hans van Kranenburg
2019-01-23 17:41         ` David Sterba
2019-01-23 18:18         ` Sasha Levin
2019-01-23 19:32           ` Hans van Kranenburg
2019-11-19 15:23             ` Ben Hutchings
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 073/117] btrfs: fix use-after-free due to race between replace start and cancel Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 074/117] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 075/117] Btrfs: fix access to available allocation bits when starting balance Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 076/117] btrfs: improve error handling of btrfs_add_link Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 077/117] tty/serial: do not free trasnmit buffer page under port lock Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 078/117] tools lib traceevent: Fix compile warnings in tools/lib/traceevent/event-parse.c Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 079/117] perf intel-pt: Fix error with config term "pt=0" Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 080/117] perf tests ARM: Disable breakpoint tests 32-bit Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 081/117] perf svghelper: Fix unchecked usage of strncpy() Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 082/117] perf parse-events: " Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 083/117] perf vendor events intel: Fix Load_Miss_Real_Latency on SKL/SKX Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 084/117] netfilter: ipt_CLUSTERIP: check MAC address when duplicate config is set Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 085/117] netfilter: ipt_CLUSTERIP: remove wrong WARN_ON_ONCE in netns exit routine Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 086/117] netfilter: ipt_CLUSTERIP: fix deadlock " Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 087/117] x86/topology: Use total_cpus for max logical packages calculation Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 088/117] dm crypt: use u64 instead of sector_t to store iv_offset Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 089/117] dm kcopyd: Fix bug causing workqueue stalls Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 090/117] perf stat: Avoid segfaults caused by negated options Sasha Levin
2019-01-08 19:25 ` [PATCH AUTOSEL 4.20 091/117] tools lib subcmd: Don't add the kernel sources to the include path Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 092/117] dm snapshot: Fix excessive memory usage and workqueue stalls Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 093/117] perf cs-etm: Correct packets swapping in cs_etm__flush() Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 094/117] perf tools: Add missing sigqueue() prototype for systems lacking it Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 095/117] perf tools: Add missing open_memstream() " Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 096/117] quota: Lock s_umount in exclusive mode for Q_XQUOTA{ON,OFF} quotactls Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 097/117] clocksource/drivers/integrator-ap: Add missing of_node_put() Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 098/117] dm: Check for device sector overflow if CONFIG_LBDAF is not set Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 099/117] Bluetooth: btusb: Add support for Intel bluetooth device 8087:0029 Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 100/117] ALSA: bebob: fix model-id of unit for Apogee Ensemble Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 101/117] sysfs: Disable lockdep for driver bind/unbind files Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 102/117] IB/usnic: Fix potential deadlock Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 103/117] scsi: mpt3sas: fix memory ordering on 64bit writes Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 104/117] scsi: smartpqi: correct lun reset issues Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 105/117] ath10k: fix peer stats null pointer dereference Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 106/117] scsi: smartpqi: call pqi_free_interrupts() in pqi_shutdown() Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 107/117] KVM: PPC: Book3S HV: Align gfn to L1 page size when inserting nest-rmap entry Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 108/117] KVM: PPC: Book3S HV: Apply combination of host and l1 pte rc for nested guest Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 109/117] scsi: megaraid: fix out-of-bound array accesses Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 110/117] iomap: don't search past page end in iomap_is_partially_uptodate Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 111/117] bnx2x: Fix NULL pointer dereference in bnx2x_del_all_vlans() on some hw Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 112/117] ocfs2: fix panic due to unrecovered local alloc Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 113/117] mm/page-writeback.c: don't break integrity writeback on ->writepage() error Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 114/117] mm/swap: use nr_node_ids for avail_lists in swap_info_struct Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 115/117] userfaultfd: clear flag if remap event not enabled Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 116/117] mm, proc: be more verbose about unstable VMA flags in /proc/<pid>/smaps Sasha Levin
2019-01-08 19:26 ` [PATCH AUTOSEL 4.20 117/117] mm/memblock.c: skip kmemleak for kasan_init() Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190108192628.121270-61-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=cai@lca.pw \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome