From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752574AbcF2B1J (ORCPT ); Tue, 28 Jun 2016 21:27:09 -0400 Received: from smtprelay0248.hostedemail.com ([216.40.44.248]:34958 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752423AbcF2B1H (ORCPT ); Tue, 28 Jun 2016 21:27:07 -0400 X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 50,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::::::,RULES_HIT:41:355:379:421:541:599:800:960:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2393:2525:2553:2560:2563:2682:2685:2859:2895:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3622:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6119:6261:7875:7903:8603:9025:10004:10400:10848:10967:11026:11232:11658:11914:12043:12296:12438:12517:12519:12555:12663:12740:13439:13618:14096:14097:14180:14181:14659:14721:21080:21325,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:error,Custom_rules:0:0:0,LFtime:2000,LUA_SUMMARY:none X-HE-Tag: class27_594c68cbe2e57 X-Filterd-Recvd-Size: 3649 Date: Tue, 28 Jun 2016 21:26:52 -0400 From: Steven Rostedt To: Minchan Kim Cc: Rabin Vincent , Namhyung Kim , , "linux-kernel@vger.kernel.org" , anderson@redhat.com Subject: Re: [QUESTION] Is there a better way to get ftrace dump on guest? Message-ID: <20160628212652.295cbb21@gandalf.local.home> In-Reply-To: <20160629005741.GA8844@bbox> References: <20160628062514.GA4674@danjae.aot.lge.com> <20160628164634.GA4202@debian> <20160629005741.GA8844@bbox> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 29 Jun 2016 09:57:41 +0900 Minchan Kim wrote: > Hello, > > On Tue, Jun 28, 2016 at 06:46:34PM +0200, Rabin Vincent wrote: > > On Tue, Jun 28, 2016 at 03:33:18PM +0900, Namhyung Kim wrote: > > > On Tue, Jun 28, 2016 at 3:25 PM, Namhyung Kim wrote: > > > > I'm running some guest machines for kernel development. For debugging > > > > purpose, I use lots of trace_printk() since it's faster than normal > > > > printk(). When kernel crash happens the trace buffer is printed on > > > > console (I set ftrace_dump_on_oops) but it takes too much time. I > > > > don't want to reduce the size of ring buffer as I want to collect the > > > > debug info as much as possible. And I also want to see trace from all > > > > cpu so 'ftrace_dump_on_oop = 2' is not an option. > > > > > > > > I know the kexec/kdump (and the crash tool) can dump and analyze the > > > > trace buffer later. But it's cumbersome to do it everytime and more > > > > importantly, I don't want to spend the memory for the crashkernel. > > > > Assuming you're using QEMU: > > > > QEMU has a dump-guest-memory command which can be used to dump the > > guest's entire memory to an ELF which can be loaded by the crash utility > > to extract the trace buffer. This doesn't require kexec/kdump or any > > other support from the guest kernel. > > Thanks for the hint. It's surely handy rather than kexec/kdump. > > A question is that it's possible to capture guest's entire memory > when guest kernel is oops? > I mean I don't want to capture alive guest but get snapshot image > when guest kernel encounters BUG_ON and see event trace from the > image. > > Anyway, I tried crashtool and load trace.so but failed to load > extension module 'trace.so' because read_string failed in > ftrace_get_event_type_name of trace.c. > Does it work with recent kernel? > > My kernel is 4.7.0-rc4-mm1. It probably needs another update. I usually send patches to David Anderson for updates. Fujitsu started that work and was maintaining it for a while, but I don't think they are anymore. I have no problem maintaining the trace.so module. If I get time tomorrow, I'll see if I can get it up to date again. -- Steve > > > > > It's apparently even possible to run QEMU with the guest memory in a > > file and load that to crash directly, although this is not something > > I've had a chance to try out myself: > > > > https://github.com/crash-utility/crash/commit/89ed9d0a7f7da4578294a492c1ad857244ce7352