[PATCH/RFC] kernel-doc: fix doc blocks and html

[PATCH/RFC] kernel-doc: fix doc blocks and html

[Randy Dunlap]

From: Randy Dunlap <randy.dunlap@oracle.com>

Cc: johannes@sipsolutions.net

Johannes Berg reports (Thanks!) that &struct names are not highlighted in
html output format when they are inside a DOC: block.

DOC: blocks were not escaped thru xml_escape() like other kernel-doc
comments were.  Fixed that.

However, that left a problem with <p> ($blankline_html) being processed
thru xml_escape(), converting it to &lt;p&gt;, which isn't good for the
generated html output (the <p> should remain unchanged), so this patch
also introduces the notion of "local" kernel-doc meta-characters
('\\\\mnemonic:'), which are converted to html just before writing the
stream to its output file.


Please report any problems that you (anyone) see in "highlighting"
in any output mode (text, man, html, xml).

Also update copyright to include me.

Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
---
 scripts/kernel-doc |   41 ++++++++++++++++++++++++++++++++---------
 1 file changed, 32 insertions(+), 9 deletions(-)

--- linux-2.6.23-rc4.orig/scripts/kernel-doc
+++ linux-2.6.23-rc4/scripts/kernel-doc
@@ -5,6 +5,7 @@ use strict;
 ## Copyright (c) 1998 Michael Zucchi, All Rights Reserved        ##
 ## Copyright (C) 2000, 1  Tim Waugh <twaugh@redhat.com>          ##
 ## Copyright (C) 2001  Simon Huggins                             ##
+## Copyright (C) 2005-2007  Randy Dunlap                         ##
 ## 								 ##
 ## #define enhancements by Armin Kuster <akuster@mvista.com>	 ##
 ## Copyright (c) 2000 MontaVista Software, Inc.			 ##
@@ -161,7 +162,7 @@ my $type_constant = '\%([-_\w]+)';
 my $type_func = '(\w+)\(\)';
 my $type_param = '\@(\w+)';
 my $type_struct = '\&((struct\s*)*[_\w]+)';
-my $type_struct_xml = '\\\amp;((struct\s*)*[_\w]+)';
+my $type_struct_xml = '\\&amp;((struct\s*)*[_\w]+)';
 my $type_env = '(\$\w+)';
 
 # Output conversion substitutions.
@@ -173,7 +174,9 @@ my %highlights_html = ( $type_constant, 
 			$type_struct_xml, "<i>\$1</i>",
 			$type_env, "<b><i>\$1</i></b>",
 			$type_param, "<tt><b>\$1</b></tt>" );
-my $blankline_html = "<p>";
+my $local_lt = "\\\\\\\\lt:";
+my $local_gt = "\\\\\\\\gt:";
+my $blankline_html = $local_lt . "p" . $local_gt;	# was "<p>"
 
 # XML, docbook format
 my %highlights_xml = ( "([^=])\\\"([^\\\"<]+)\\\"", "\$1<quote>\$2</quote>",
@@ -391,17 +394,19 @@ sub output_highlight {
 #	confess "output_highlight got called with no args?\n";
 #   }
 
+    if ($output_mode eq "html") {
+	$contents = local_unescape($contents);
+	# convert data read & converted thru xml_escape() into &xyz; format:
+	$contents =~ s/\\\\\\/&/g;
+    }
 #   print STDERR "contents b4:$contents\n";
     eval $dohighlight;
     die $@ if $@;
-    if ($output_mode eq "html") {
-	$contents =~ s/\\\\//;
-    }
 #   print STDERR "contents af:$contents\n";
 
     foreach $line (split "\n", $contents) {
 	if ($line eq ""){
-	    print $lineprefix, $blankline;
+	    print $lineprefix, local_unescape($blankline);
 	} else {
 	    $line =~ s/\\\\\\/\&/g;
 	    if ($output_mode eq "man" && substr($line, 0, 1) eq ".") {
@@ -1752,7 +1757,13 @@ sub process_state3_type($$) {
     }
 }
 
-# replace <, >, and &
+# xml_escape: replace <, >, and & in the text stream;
+#
+# however, formatting controls that are generated internally/locally in the
+# kernel-doc script are not escaped here; instead, they begin life like
+# $blankline_html (4 of '\' followed by a mnemonic + ':'), then these strings
+# are converted to their mnemonic-expected output, without the 4 * '\' & ':',
+# just before actual output; (this is done by local_unescape())
 sub xml_escape($) {
 	my $text = shift;
 	if (($output_mode eq "text") || ($output_mode eq "man")) {
@@ -1764,6 +1775,18 @@ sub xml_escape($) {
 	return $text;
 }
 
+# convert local escape strings to html
+# local escape strings look like:  '\\\\menmonic:' (that's 4 backslashes)
+sub local_unescape($) {
+	my $text = shift;
+	if (($output_mode eq "text") || ($output_mode eq "man")) {
+		return $text;
+	}
+	$text =~ s/\\\\\\\\lt:/</g;
+	$text =~ s/\\\\\\\\gt:/>/g;
+	return $text;
+}
+
 sub process_file($) {
     my $file;
     my $identifier;
@@ -1903,7 +1926,7 @@ sub process_file($) {
 	} elsif ($state == 4) {
 		# Documentation block
 		if (/$doc_block/) {
-			dump_section($section, $contents);
+			dump_section($section, xml_escape($contents));
 			output_intro({'sectionlist' => \@sectionlist,
 				      'sections' => \%sections });
 			$contents = "";
@@ -1923,7 +1946,7 @@ sub process_file($) {
 		}
 		elsif (/$doc_end/)
 		{
-			dump_section($section, $contents);
+			dump_section($section, xml_escape($contents));
 			output_intro({'sectionlist' => \@sectionlist,
 				      'sections' => \%sections });
 			$contents = "";

Re: kernel-doc: fix doc blocks and html

[Randy Dunlap]

On Fri, 31 Aug 2007 20:58:45 -0700 Randy Dunlap wrote:

> From: Randy Dunlap <randy.dunlap@oracle.com>
> 
> Cc: johannes@sipsolutions.net
> 
> Johannes Berg reports (Thanks!) that &struct names are not highlighted in
> html output format when they are inside a DOC: block.
> 
> DOC: blocks were not escaped thru xml_escape() like other kernel-doc
> comments were.  Fixed that.


Johannes is using a feature of kernel-doc that I wasn't even familiar
with, also one that no one else is using.  I'm sure that Johannes can
point us to some source code and generated output for it though.

If you want to see a little of it in source code form, 2 net drivers
use it, but then they aren't processed by kernel-doc for generated
output.  They are drivers/net/3c501.c & 3c527.c.  Look for the
"DOC:" comment blocks.

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: kernel-doc: fix doc blocks and html

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On Fri, 2007-08-31 at 21:06 -0700, Randy Dunlap wrote:

> Johannes is using a feature of kernel-doc that I wasn't even familiar
> with, also one that no one else is using.  I'm sure that Johannes can
> point us to some source code and generated output for it though.

It's not been merged yet:
http://johannes.sipsolutions.net/files/mac80211.h.txt
and just generated with your updated kernel-doc:
http://johannes.sipsolutions.net/files/mac80211.html

Example for the problem is in the "Hardware crypto acceleration"
section, but it's now obviously no longer visible.

> If you want to see a little of it in source code form, 2 net drivers
> use it, but then they aren't processed by kernel-doc for generated
> output.

Yeah, that's something I too need to work on. I'm thinking of making a
new include statement for the DOC: sections too, e.g.
   !P<filename>,<sectionname>

The thing is that if I simply add the docs to a docbook instead of DOC:
sections nobody will read it and I surely will continually need to point
to that. This way, people who use it daily will probably read it in the
header file but we can still generate pretty output for other places.

Anyhow, thanks a lot for this patch, the changes seem to work perfectly.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]