[gentoo-dev] markdown docs like README.md

[gentoo-dev] markdown docs like README.md

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I wonder if it would make any sense to take the effort to convert
markdown docs to html format before installing them.

I see two possibilities:

1. Create one or two eutils functions like
"domd": will go through all viable implementations like
markdown/markdown_py/Markdown.pl, convert the file and dohtml it. Is
no impelementation installed on the system it will fall back to plain
dodoc.
Optionally we could provide "md_depend" (like DEPEND="$(md_depend)")
to ensure that at least one implementation is present (might be useful
for very huge docs that are easier to read as html).

2. Introduce something like virtual/markdown... unfortunately the
"markdown" name is already reserved by app-text/discount, so I don't
really know how that would work out. And the supported arguments might
differ...

This is low priority, so if this is going to take more then little
work, I'll probably lose interest.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSQdCtAAoJEFpvPKfnPDWzTAUH/iv42uJz9S1Ky0j6jSYYNlYo
MGPdo6oQFMVHUOSBuC0tqz/k/sWnk32K3MYYoTfneVK907jMjGK+TLrdAy35nXjD
PKDQeu3lZvA9Yd1RhDtKWAKOj32OJTvxtBakX4Q+RielBhgesIzfJVAYVAFGk/l7
igwpabGuKCz8xC13DSvvMScHSod9f/eGuHGcT/4TM5DuMXodWKHuqN9Smy02cPuL
9AluDpmbRcummpyTiWjMgAvPEPhlowsg/DMSzd8Cz/eeeOJsZtIFF+fBQOGP5xHI
eIW4SoRFGnBMB8lM35aEBoN7WlrvR6lKmQay/CVqN2a3H0Zyd8Zt1p66pkkx2So=
=1R0w
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Ciaran McCreesh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 24 Sep 2013 19:49:33 +0200
hasufell <hasufell@gentoo.org> wrote:
> I wonder if it would make any sense to take the effort to convert
> markdown docs to html format before installing them.

Aren't there thirty seven different incompatible formats all called
"markdown", all of which require different tools to process correctly?

- -- 
Ciaran McCreesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.21 (GNU/Linux)

iEYEARECAAYFAlJB0lQACgkQ96zL6DUtXhGGawCgr5uPYjBOTIYyxz1/zUKJCUpg
b8oAoMMFn4F1psgiy10rqUNqCJeF1HU3
=jfdj
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 24 Sep 2013 19:49:33 +0200
hasufell <hasufell@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I wonder if it would make any sense to take the effort to convert
> markdown docs to html format before installing them.

Converting them to HTML format is useful for people whom do not want
to use a Markdown viewer and would benefit from reading them in HTML
format, as opposed to text mangled with formatting constructs.

Do we want to cover this by an USE flag or apply the conversion for
every package based on a present implementation? What if someone
decides to switch between both? Do we want to keep the original
Markdown file installed as well or would we prune it after conversion?

> I see two possibilities:
> 
> 1. Create one or two eutils functions like
> "domd": will go through all viable implementations like
> markdown/markdown_py/Markdown.pl, convert the file and dohtml it. Is
> no impelementation installed on the system it will fall back to plain
> dodoc.

Treating files as Markdown sounds like a good start.

> Optionally we could provide "md_depend" (like DEPEND="$(md_depend)")
> to ensure that at least one implementation is present (might be useful
> for very huge docs that are easier to read as html).

No idea if such construct is the right way; I generally don't like this
when other already implemented approaches exist, like using a virtual.

Why wouldn't a virtual suffice?

> 2. Introduce something like virtual/markdown... unfortunately the
> "markdown" name is already reserved by app-text/discount, so I don't
> really know how that would work out. And the supported arguments might
> differ...

A virtual for converters sounds better to me than $(md_depend); it
spares out making eutils more complex, having to inherit eutils and
having the dependencies where one would not expect them.

(Do we want a virtual for viewers too?)

> This is low priority, so if this is going to take more then little
> work, I'll probably lose interest.

It's a small change you want to see; it'll take little time if we go
for implementing just what is really necessary, the KISS principle.

- -- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.21 (GNU/Linux)

iQEcBAEBAgAGBQJSQe92AAoJEJWyH81tNOV9+7kIAIMJVjDElGCbvib0v8YiFI7X
IKygjcTosHvim9tjN78ShXn+1RvCyPeDZVEGU4+tQ3mYOiHe/mOm5QC6N/tN/ZOy
+Vl42Xfwd6QGmObuJHgsMcpPQmjx3GUDKSIq0j3tZfx6MSVz9/QWtXV+h+7ZZo/W
jj/lLhc0BQ5ryf+aoB/poutANoiPL0QzxTOpPZ4v/MqY8SUN/Pv7V27kdPdgYzut
HkxvWUmpA+Vy8483X8IQPVQxaITv0e+O9v0Zjh0n5yc14qxx1ChlzIN2rfntky8J
JHJRwN6gsUhC6eZdxoOUgkOUlRi523tlAEp7on4b0WmBpNSAS/aF14x8Oaxi/kQ=
=ejC9
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

Dnia 2013-09-24, o godz. 19:49:33
hasufell <hasufell@gentoo.org> napisał(a):

> I wonder if it would make any sense to take the effort to convert
> markdown docs to html format before installing them.

What for? The point of markups like Markdown is for the text to be
readable as plain text. Converting it to HTML goes against
this principle.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/24/2013 10:15 PM, Michał Górny wrote:
> Dnia 2013-09-24, o godz. 19:49:33 hasufell <hasufell@gentoo.org>
> napisał(a):
> 
>> I wonder if it would make any sense to take the effort to
>> convert markdown docs to html format before installing them.
> 
> What for? The point of markups like Markdown is for the text to be 
> readable as plain text. Converting it to HTML goes against this
> principle.
> 

I am not sure if I can follow that logic. The purpose is to not
convert it to html?

I am very well aware that it is readable both ways, but some users
were requesting to convert installed docs in markdown format into
html. I personally don't care much.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSQfkDAAoJEFpvPKfnPDWzTp4H/0ohVRG1VTzLqDgrW8B8AgPP
2z6LPD1O1lkdzjT0rTEmEFmJmFD4IDISXv3PrXXxiN+V2purrz/0cR91so6BHaHJ
vMSt4pmfPxyiH7xvEYu0P89iK5HsAtsJEnbrF6D7dvWvNyMt4DuoujgMROeanFjS
DGmarLCDlBkgxi/Ym09E/TzBR9J40FW2sc1MxMHABRlHySHqsc1AxCwGxEgit+Uk
SuNV4roSDTi51aYSOA/X3ZJqP1jtAtlq9Lg6pHvaJTl8vorj1rqwlkXk1hlR1txC
zh/zhilp9g//+n7kvzLUCaSUXSWURbNPdAqcfXn7BxyqzrDmFr/ggDiP2WwuGbA=
=c3gv
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 24 Sep 2013 18:56:33 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> hasufell <hasufell@gentoo.org> wrote:
> > I wonder if it would make any sense to take the effort to convert
> > markdown docs to html format before installing them.
> 
> Aren't there thirty seven different incompatible formats all called
> "markdown", all of which require different tools to process correctly?

Yeah, there are. But do we need to support them all? A lot of these
formats are used on websites alone; so, they are not relevant here. If
you boil it down to formats you store, use or process locally, it gets
shorter. Do we actually want to process all the alternatives correctly?

=======================================================================
TL;DR: Lack of standardization is a mess, let's work with what we have.
=======================================================================

When I hear Markdown I think about where it all started:

    http://daringfireball.net/projects/markdown/

And I would propose we focus on that syntax to start with; if we want
to support a widely used alternative, we could implement support for
specifying which format is used and use a compatible converter.

We then come down to a popular alternative:

    https://help.github.com/articles/github-flavored-markdown

Yes, I can see what you mean, it leads to certain inconsistencies; and
David Greenspan (not the actor; founder of EtherPad, now working
on Meteor) also noticed that about a year ago. He reached out to Jeff
Atwood which shared the same thought and reached out to a lot of people:

    http://www.codinghorror.com/blog/2012/10/the-future-of-markdown.html

But apparently, this doesn't seem to have worked out so well; this is
because it is often used within the context of a website and the
Markdown contents aren't really exchanged among websites.

They are not really trying to standardize a protocol here, but rather
something people use when writing out a comment on some website; if
users then switch from Stack Overflow to Reddit they will barely tell
the differences in the particular behavior the syntax would produce.

GitHub is kind of an exception here, they allow you to place files in
your repository that you can read on their site or read on your device
as plain-text, using a Markdown viewer or a converter.

While a Markdown viewer has existed before that and you could just do
it; I don't recall it being common use in the back of the days, it's
only when you plan to exchange it across communities and / or people
that a standard becomes a necessity. GitHub has introduced exchanging.

But, are there other alternatives? Yes, there is the original; some
people might be writing their repository's README.md in that format,
unintended, then not noticing that GitHub reads it a bit differently.

It's going to be hard, but I think we might want to let the user decide
which particular implementation is in favor; or give the user some way
to switch back in forth, or let the user have the choice to use some
converter that attempts to implement the best of each format.

As long as websites use Markdown in their own way; we will be unable to
use a standard, let alone ensure that all the Markdown files we get to
read are in a particular standard for Markdown files in repositories.

Not until the big Markdown consumers step up and change the game.

Let's work with what we are given for now; but indeed, we should be
well aware that no standard is present and the future can change.

- -- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.21 (GNU/Linux)

iQEcBAEBAgAGBQJSQgVRAAoJEJWyH81tNOV9lssIALs7e95SC7XnCZ2KpZlFSYvI
GqqIe8CrXrGn4f+xugmgSQgDc3QqNIxTj5lib/7K3pMOtZfqxwGsNe+nVWvlQ0sH
3jhUFIGhlcGdnBt+ibkkfrdyFmPa2lKXKzsK/XLzR2on0AezspeHtsWEfQzNLjBn
rgMOXDrIwHGgdOUKAerKW//6CJUhvvn6M9gxFImiLcQueKMNLxHBYm09dkbRU+XM
1/Yv5HhvqwbnVOtDIR9up736D3lv02ZmGg0kxvf7EjEJT+1TJDoS7kezamSXk+1o
xflwOLGcEURkRTQj39I9qZCO21yBnw6Ge/JDtNn6Tw94KysueTN/u7rtlPp8/rQ=
=MpAx
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

On Tue, 24 Sep 2013 22:15:15 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-09-24, o godz. 19:49:33
> hasufell <hasufell@gentoo.org> napisał(a):
> 
> > I wonder if it would make any sense to take the effort to convert
> > markdown docs to html format before installing them.
> 
> What for? The point of markups like Markdown is for the text to be
> readable as plain text. Converting it to HTML goes against
> this principle.

You are pulling things out of context. Let me quote [1]:

> Thus, “Markdown” is two things: (1) a plain text formatting syntax;
> and (2) a software tool, written in Perl, that converts the plain
> text formatting to HTML.

(1) If it is a formatting syntax, then why would we make text any
more readable than it already is; actually, by mangling it with
formatting constructs it could even end up being less readable.

(2) It is a tool as well, saying it goes against a principle would be
denying its existence and usefulness; consider that the global use of
Markdown consists of a lot of conversion, we shouldn't neglect that
people want to use it as it benefits them.

Principle or not, we shouldn't force our users down a particular way to
reading them; as that's not what our meta-distribution stands for.

 [1]: http://daringfireball.net/projects/markdown/

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Dnia 2013-09-24, o godz. 23:50:20
Tom Wijsman <TomWij@gentoo.org> napisał(a):

> On Tue, 24 Sep 2013 22:15:15 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > Dnia 2013-09-24, o godz. 19:49:33
> > hasufell <hasufell@gentoo.org> napisał(a):
> > 
> > > I wonder if it would make any sense to take the effort to convert
> > > markdown docs to html format before installing them.
> > 
> > What for? The point of markups like Markdown is for the text to be
> > readable as plain text. Converting it to HTML goes against
> > this principle.
> 
> Principle or not, we shouldn't force our users down a particular way to
> reading them; as that's not what our meta-distribution stands for.

We shouldn't either work on working for every fancy a single user may
have. If there's a real benefit from converting markdown to HTML,
please show me one.

As far as I can see it, we're either talking about:

1) replacing semi-readable Markdown with unreadable HTML that will
require special tools for proper display,

2) installing duplicate files (the same data in markdown and in HTML),

3) adding some more ugly awful magic that will make binary packages
even less useful.

That said, I'd rather see people using *tools* to display Markdown
rather than converting everything 90s-style.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[heroxbd]

hasufell <hasufell@gentoo.org> writes:

> I wonder if it would make any sense to take the effort to convert
> markdown docs to html format before installing them.

I'd rather leave it alone, as markdown is more readable than html, IMHO.

Anyway, when we read html in the console, it's converted back to plan text.

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 2977 bytes --]

On Wed, 25 Sep 2013 00:07:15 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-09-24, o godz. 23:50:20
> Tom Wijsman <TomWij@gentoo.org> napisał(a):
> 
> > Principle or not, we shouldn't force our users down a particular
> > way to reading them; as that's not what our meta-distribution
> > stands for.
> 
> We shouldn't either work on working for every fancy a single user may
> have. If there's a real benefit from converting markdown to HTML,
> please show me one.

It brings them to the same structure as any other HTML documentation;
if I'm going to take a complete opposite idea, we could also consider
converting PDFs to HTML for people whom don't want a PDF reader or
want to read the documentation on another device that doesn't even
support PDF. Furthermore, HTML and CSS gives them control on how the
text should be formatting; something both PDF and Markdown does not.

PDF is of course not the best example, because the binary of a PDF isn't
really readable to a human whereas Markdown was meant to stay readable;
but I'm just saying that whereas you might not see a benefit, that
doesn't mean that nobody else does.

If I want to browse all documentation with a browser, just going to
something like file:///usr/share/doc and read them all in the same
style (using an extension like Stylebot or so); then it would be neat
if Gentoo could bring them down to the same format to me, instead of
that I would need to do local conversion or use multiple viewers for
what could be in one canonical place.

Why do I need a browser, a PDF reader and a Markdown viewer and
possibly more clients to read my documentation in a formatted way?

> As far as I can see it, we're either talking about:
> 
> 1) replacing semi-readable Markdown with unreadable HTML that will
> require special tools for proper display,

Just some basic CSS will do just fine.

> 2) installing duplicate files (the same data in markdown and in HTML),

This hasn't been discussed yet; but it doesn't need to, it's the usual
INSTALL_MASK story.

> 3) adding some more ugly awful magic that will make binary packages
> even less useful.

For binary packages a choice has to be made; trying to solve things for
binary packages is like discussing something to be implemented on a
binary distro, you simply can't bring the usefulness we are discussing
here to a binary package because of its nature.

> That said, I'd rather see people using *tools* to display Markdown
> rather than converting everything 90s-style.

I'd rather have a single tool that displays documentation and display
it really well; people are still converting things these days, they
will continue to do so in the future. Some things aren't compatible.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 2648 bytes --]

On 25 September 2013 10:30, Tom Wijsman <TomWij@gentoo.org> wrote:

> If I want to browse all documentation with a browser, just going to
> something like file:///usr/share/doc and read them all in the same
> style (using an extension like Stylebot or so); then it would be neat
> if Gentoo could bring them down to the same format to me, instead of
> that I would need to do local conversion or use multiple viewers for
> what could be in one canonical place.
>


How about doing it "Out of band" by a yet-to-exist gentoo tool that can
create a format to the users liking on demand?

I mean, whatever way you look at it, such a tool *MUST* exist to simply
format the markdown to X-Format anyway.

So why do this during ebuild phases? Why not have a tool that handles this?

This greatly simplifies the problem that will transpire if we want to
support more than one markdown standard, namely, an increased bulk of
dependencies.

As to identifying what standard to use for a given markdown document, I
feel trying to do that automatically will result in sorrow, as will trying
to develop one tool that handles all formats in the same document.

I think maybe we could support a metafile of some description in the
doc/<$pn-$pv>/ directory that describes a list of documents and the
relevant format decoder to use for that document.

This approach means we can do this for more than just markdown documents,
and we can have support of extension-free files that happen to be in RST
format instead, or ascii-doc, or whatever, ie:

in /doc/$P/formats.meta
README.md : markdown/gfm

and then any of our tools can translate on demand to the format needed:

less $PATH/README.md

^ could even be hooked to discover formats.meta and run the content through
a ANSI formatter to translate bold indications into escape codes.

And the same tool could be used as a backend for whatever web-browser
centric documentation viewer we provide to render the files as html, or
even do things like

gentoo-fm $PATH/README.md --formatter pdf > /tmp/doc.pdf && okular
/tmp/doc.pdf

In the event somebody wants to read a simple markdown document as PDF.

( This also has the benefits of not necessarily adding a large amount of
cruft to doc/ for people who don't need  the documentation, and means they
wont have to rebuild the package *JUST* to get documentation in a different
format )


TL;DR - We can provide a tool, it doesn't have to be locked in to the
ebuild phase to be useful, and could be more useful to be seperate of
ebuild systems, with ebuild systems only providing minimal amounts of
context to help a tool know how to process the document.





-- 
Kent

[-- Attachment #2: Type: text/html, Size: 3792 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 5766 bytes --]

On Wed, 25 Sep 2013 11:27:50 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 25 September 2013 10:30, Tom Wijsman <TomWij@gentoo.org> wrote:
> 
> > If I want to browse all documentation with a browser, just going to
> > something like file:///usr/share/doc and read them all in the same
> > style (using an extension like Stylebot or so); then it would be
> > neat if Gentoo could bring them down to the same format to me,
> > instead of that I would need to do local conversion or use multiple
> > viewers for what could be in one canonical place.
> >
> 
> 
> How about doing it "Out of band" by a yet-to-exist gentoo tool that
> can create a format to the users liking on demand?
> 
> I mean, whatever way you look at it, such a tool *MUST* exist to
> simply format the markdown to X-Format anyway.

That sounds like a converter.

> So why do this during ebuild phases? Why not have a tool that handles
> this?

Because we are discussing conversion in an ebuild context.

> This greatly simplifies the problem that will transpire if we want to
> support more than one markdown standard, namely, an increased bulk of
> dependencies.

The existence of a tool does not exclude that an ebuild cannot use it;
so, I agree with your paragraph but that doesn't necessarily mean we can
apply this in an ebuild context.

> As to identifying what standard to use for a given markdown document,
> I feel trying to do that automatically will result in sorrow, as will
> trying to develop one tool that handles all formats in the same
> document.

Yes, automatic detection and applying a whole conversion based on that
detection would be bad; writing something that only deals with the
common elements, would be more neat. 

The question being whether we want to focus on the original Markdown or
the GitHub Flavored Markdown to base ourselves on.

> I think maybe we could support a metafile of some description in the
> doc/<$pn-$pv>/ directory that describes a list of documents and the
> relevant format decoder to use for that document.

Why does documentation generation and/or conversion so suddenly be done
by the user? it needlessly complicates things, especially since the
user now has to store those generated or converted in a separate
location; this is not the way it has been done and I don't understand
why we need any change in that.

The maintainer could specify the format in the ebuild; for bugs where
it has found to be of a different format, this is an easy fix.

> This approach means we can do this for more than just markdown
> documents, and we can have support of extension-free files that
> happen to be in RST format instead, or ascii-doc, or whatever, ie:

This is no longer a discussion with regard to Markdown; but a whole
topic on its own, at least if you want to be pursue consistency it
might be worth bringing this up its own thread (or at least change the
subject to denote it is no longer about Markdown in specific).

> and then any of our tools can translate on demand to the format
> needed:
>
> ^ could even be hooked to discover formats.meta and run the content
> through a ANSI formatter to translate bold indications into escape
> codes.

My browser can't; so, I'll need conversion.
 
> And the same tool could be used as a backend for whatever web-browser
> centric documentation viewer we provide to render the files as html,
> or even do things like

What kind of backend would this be? A web server daemon?

> gentoo-fm $PATH/README.md --formatter pdf > /tmp/doc.pdf && okular
> /tmp/doc.pdf
> 
> In the event somebody wants to read a simple markdown document as PDF.

But why should we go through hard hoops to read something simple?

Why not just have it installed and opened in the format the user wants?

> ( This also has the benefits of not necessarily adding a large amount
> of cruft to doc/ for people who don't need  the documentation, and
> means they wont have to rebuild the package *JUST* to get
> documentation in a different format )

That doesn't happen as these are controlled with USE flags; eg, USE=pdf.

If we plan on introducing more documentation formats, we might want to
introduce an USE expand to make it easier for the user to configure;
instead of the local USE flags we are using now.

> TL;DR - We can provide a tool, it doesn't have to be locked in to the
> ebuild phase to be useful,

Tools used by ebuilds and eclasses can usually be used outside that
context as well; it isn't really locked, and its usefulness doesn't
change depending on where it is used. It's rather what the addition of
the context you place it in that matters here.

> and could be more useful to be seperate of ebuild systems,

Perhaps it could be, but I don't see why; it feels less handy to do.

> with ebuild systems only providing minimal amounts of
> context to help a tool know how to process the document.

If you put the tool in the context of the user, maintainers will not
really be aware of or contributing to the context; eg. formats.meta.
This would put the format responsibility in the hands of the user...

Whereas if it was in an ebuild, it could be a matter of changing `domd
README.md` to `domd -gfm README.md` (or if gfm defaults, to `domd -orig
README.md`). No separate file, barely any extra characters; easy to fix.

And as a result, documentation files in the format that the user wants.

"Here, format X you don't like." --> "Which format would you like?"

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On 25 September 2013 12:33, Tom Wijsman <TomWij@gentoo.org> wrote:

> The existence of a tool does not exclude that an ebuild cannot use it;
> so, I agree with your paragraph but that doesn't necessarily mean we can
> apply this in an ebuild context.
>

I guess I can agree I would be amenable to a scenario where you could do
either.

Gentoo is about choices, and if its viable that we provide the choice to
generate the alternative formats during build so that a user doesn't have
to invoke a command to generate the alternative format later, then why not?

Recipients who don't want the overhead of disk cruft can just omit the
relevant useflags and get only documentation in its source form ( with
maybe a hints file generated by the domd commands instead of performing the
full translation )

Just I should make a strong suggestion that such alternative formats should
be generated during src_build , *NOT* src_install, mostly due to the
differences in privilege separation under userpriv, a problem that has
caused at least one bug in the past (
https://bugs.gentoo.org/show_bug.cgi?id=454058 )


-- 
Kent

[-- Attachment #2: Type: text/html, Size: 1785 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Martin Gysel]

Am 24.09.2013 19:49, schrieb hasufell:
> I wonder if it would make any sense to take the effort to convert
> markdown docs to html format before installing them.
> 

following that logic we should also consider converting all man and info
pages to html...

it doesn't make any sense... simply provide or suggest a markdown
capable viewer to the user

/martin

Re: [gentoo-dev] markdown docs like README.md

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1997 bytes --]

Dnia 2013-09-25, o godz. 00:30:27
Tom Wijsman <TomWij@gentoo.org> napisał(a):

> On Wed, 25 Sep 2013 00:07:15 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> Why do I need a browser, a PDF reader and a Markdown viewer and
> possibly more clients to read my documentation in a formatted way?

And why do I need a special HTML-formatting tool (which is much more
expensive) to read documentation in any way? Or rather, to drop all
the useless formatting.

> > As far as I can see it, we're either talking about:
> > 
> > 1) replacing semi-readable Markdown with unreadable HTML that will
> > require special tools for proper display,
> 
> Just some basic CSS will do just fine.

And how does that help with 'cat' output? Or vim? Or many other
standard *console* tools Gentoo users use.

> > 2) installing duplicate files (the same data in markdown and in HTML),
> 
> This hasn't been discussed yet; but it doesn't need to, it's the usual
> INSTALL_MASK story.

And how does this distinguish between HTML cruft converted from
Markdown and HTML-only docs?

> > 3) adding some more ugly awful magic that will make binary packages
> > even less useful.
> 
> For binary packages a choice has to be made; trying to solve things for
> binary packages is like discussing something to be implemented on a
> binary distro, you simply can't bring the usefulness we are discussing
> here to a binary package because of its nature.

Which is not reason to make it even worse.

> > That said, I'd rather see people using *tools* to display Markdown
> > rather than converting everything 90s-style.
> 
> I'd rather have a single tool that displays documentation and display
> it really well; people are still converting things these days, they
> will continue to do so in the future. Some things aren't compatible.

It's called 'less'. Open a bug against it, ask our devs to include
a formatter in 'lesspipe'. Tadaam!

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 3436 bytes --]

On Wed, 25 Sep 2013 09:57:26 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-09-25, o godz. 00:30:27
> Tom Wijsman <TomWij@gentoo.org> napisał(a):
> 
> > On Wed, 25 Sep 2013 00:07:15 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > Why do I need a browser, a PDF reader and a Markdown viewer and
> > possibly more clients to read my documentation in a formatted way?
> 
> And why do I need a special HTML-formatting tool (which is much more
> expensive) to read documentation in any way? Or rather, to drop all
> the useless formatting.

Who says you need to?

We're not suggesting completely replacing what we currently have as far
as I am aware of; so, if you don't want to see a change in what is
installed for you, you'll be able to keep it that way under what is
being suggested in this thread. Choice implies that you can keep things
the way they have been.

> > > As far as I can see it, we're either talking about:
> > > 
> > > 1) replacing semi-readable Markdown with unreadable HTML that will
> > > require special tools for proper display,
> > 
> > Just some basic CSS will do just fine.
> 
> And how does that help with 'cat' output? Or vim? Or many other
> standard *console* tools Gentoo users use.
>
> > > 2) installing duplicate files (the same data in markdown and in
> > > HTML),
> > 
> > This hasn't been discussed yet; but it doesn't need to, it's the
> > usual INSTALL_MASK story.
> 
> And how does this distinguish between HTML cruft converted from
> Markdown and HTML-only docs?

You just choose to not use a converter. No problem here.

> > > 3) adding some more ugly awful magic that will make binary
> > > packages even less useful.
> > 
> > For binary packages a choice has to be made; trying to solve things
> > for binary packages is like discussing something to be implemented
> > on a binary distro, you simply can't bring the usefulness we are
> > discussing here to a binary package because of its nature.
> 
> Which is not reason to make it even worse.

Neither is it a reason to stop progress.

Looking at the mailing list history, I could use this binary package
argument on many threads. People that want binary packages should live
with a default under the current implementation; as for a possible
future implementation, that could possibly have split up tarballs so we
can selectively install parts of the binary package such that this would
not be a problem at all. Discussing binary packages here is off-topic.

> > > That said, I'd rather see people using *tools* to display Markdown
> > > rather than converting everything 90s-style.
> > 
> > I'd rather have a single tool that displays documentation and
> > display it really well; people are still converting things these
> > days, they will continue to do so in the future. Some things aren't
> > compatible.
> 
> It's called 'less'. Open a bug against it, ask our devs to include
> a formatter in 'lesspipe'. Tadaam!

Exactly, now this thread wants to make alternatives to that possible;
just because one tool exists doesn't mean everyone wants to use it,
there is no one size fits all solution. That's where choice comes from.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]

On Wed, 25 Sep 2013 08:36:56 +0200
Martin Gysel <m.gysel@gmail.com> wrote:

> Am 24.09.2013 19:49, schrieb hasufell:
> > I wonder if it would make any sense to take the effort to convert
> > markdown docs to html format before installing them.
> > 
> 
> following that logic we should also consider converting all man and
> info pages to html...
> 
> it doesn't make any sense...

Yet there are many websites that show these converted man pages, as well
as users looking them up there; that it doesn't make sense for you
doesn't necessarily mean that nobody else is doing it.

> simply provide or suggest a markdown
> capable viewer to the user

Yet another viewer I have to install; imagine that I had to do the same
for video formats, then I'd have to install an AVI player, a MKV
player, a MP4 player, a MOV player, an OGG player, a VOB player, ...

No, I'd rather have a single converted format and/or a single player.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[hasufell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I lost interest.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSQttjAAoJEFpvPKfnPDWzRXYIAIQKnHSRoChJ/gHd56KxaaW+
KGYQG9xW/OAO68qiubRW/1YK6vMRfWk9k/6m/pwwIPmFfYBAucT6jwBkfnR/HgMD
TB08WpbooFTYGqX378jB07do0SNoYiQCNOCFJ313K57yAU7fCc5+VdMTM8v9e/i5
M94snDehz7LCTjXT0Q+MH9InWaOT4aHXPKhr6SZZUsR+MjtKhWcL8JjeHZ78mLFt
/Ph33fEur3HOAPDJ5s3u+AHeOfKVuBPmGSKI8GfY88XatcYqapaLYIZKfTxIp0Ip
b0KKVxQ3vB2o9+MwCMfKmx+Cpyp3eklh/fOUusxQIqGcHccrRhtvK6r7gp8LFRc=
=LnLn
-----END PGP SIGNATURE-----

Re: [gentoo-dev] markdown docs like README.md

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2423 bytes --]

Dnia 2013-09-25, o godz. 14:29:52
Tom Wijsman <TomWij@gentoo.org> napisał(a):

> > > > 3) adding some more ugly awful magic that will make binary
> > > > packages even less useful.
> > > 
> > > For binary packages a choice has to be made; trying to solve things
> > > for binary packages is like discussing something to be implemented
> > > on a binary distro, you simply can't bring the usefulness we are
> > > discussing here to a binary package because of its nature.
> > 
> > Which is not reason to make it even worse.
> 
> Neither is it a reason to stop progress.

Excuse me but *how* is this related to progress at all? You're talking
about converting *newer* format files to *older* format that will
require special processing for display anyway.

Worse than that, you are actually talking about doing the conversion
*on files*, that is storing duplicate data. I'd expect progress to go
*forward*. Introducing compatibility files for reading non-mandatory
files using a web browser doesn't sound anywhere near progress.

> > > > That said, I'd rather see people using *tools* to display Markdown
> > > > rather than converting everything 90s-style.
> > > 
> > > I'd rather have a single tool that displays documentation and
> > > display it really well; people are still converting things these
> > > days, they will continue to do so in the future. Some things aren't
> > > compatible.
> > 
> > It's called 'less'. Open a bug against it, ask our devs to include
> > a formatter in 'lesspipe'. Tadaam!
> 
> Exactly, now this thread wants to make alternatives to that possible;
> just because one tool exists doesn't mean everyone wants to use it,
> there is no one size fits all solution. That's where choice comes from.

And what benefits do those 'alternatives' give us? Featurism, that's
all. Implementing new features for the sake of doing something. Someone
throws a random idea, let's implement it for the sake of choice.

Seriously, how many people actually *care* about reading /usr/share/doc
with a HTML browser? How many people actually need it? That is, how
many people get real benefit rather than shiny formatting in their
favorite tool.

Gentoo is not about bending everything upstream provides to match every
tool a particular user likes. Improving the tools give more benefit
than pushing compatibility cruft.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

Dnia 2013-09-25, o godz. 14:38:14
Tom Wijsman <TomWij@gentoo.org> napisał(a):

> On Wed, 25 Sep 2013 08:36:56 +0200
> Martin Gysel <m.gysel@gmail.com> wrote:
> 
> > Am 24.09.2013 19:49, schrieb hasufell:
> > > I wonder if it would make any sense to take the effort to convert
> > > markdown docs to html format before installing them.
> > > 
> > 
> > following that logic we should also consider converting all man and
> > info pages to html...
> > 
> > it doesn't make any sense...
> 
> Yet there are many websites that show these converted man pages, as well
> as users looking them up there; that it doesn't make sense for you
> doesn't necessarily mean that nobody else is doing it.

*Websites* is the keyword here.

> > simply provide or suggest a markdown
> > capable viewer to the user
> 
> Yet another viewer I have to install; imagine that I had to do the same
> for video formats, then I'd have to install an AVI player, a MKV
> player, a MP4 player, a MOV player, an OGG player, a VOB player, ...
> 
> No, I'd rather have a single converted format and/or a single player.

Yet you don't mind most of our users installing a Markdown converter
(and possibly even more converters) in the sake of converting docs to
the format you like...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 4453 bytes --]

On Wed, 25 Sep 2013 20:07:42 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-09-25, o godz. 14:29:52
> Tom Wijsman <TomWij@gentoo.org> napisał(a):
> 
> > > > > 3) adding some more ugly awful magic that will make binary
> > > > > packages even less useful.
> > > > 
> > > > For binary packages a choice has to be made; trying to solve
> > > > things for binary packages is like discussing something to be
> > > > implemented on a binary distro, you simply can't bring the
> > > > usefulness we are discussing here to a binary package because
> > > > of its nature.
> > > 
> > > Which is not reason to make it even worse.
> > 
> > Neither is it a reason to stop progress.
> 
> Excuse me but *how* is this related to progress at all?

Progress in providing choice, helping to support a single viewer.

> You're talking
> about converting *newer* format files to *older* format that will
> require special processing for display anyway.

The age or functionality of a format is not what we should discuss here
as it does not matter when talking about this progress.

> Worse than that, you are actually talking about doing the conversion
> *on files*, that is storing duplicate data.

Only if one chooses to do so, which hasn't even been decided yet.

> I'd expect progress to go *forward*. Introducing compatibility files
> for reading non-mandatory files using a web browser doesn't sound
> anywhere near progress.

That depends on how you define what is being progressed in; also if you
don't want to see compatibility, then yes, it is not progress for you.

> > > > > That said, I'd rather see people using *tools* to display
> > > > > Markdown rather than converting everything 90s-style.
> > > > 
> > > > I'd rather have a single tool that displays documentation and
> > > > display it really well; people are still converting things these
> > > > days, they will continue to do so in the future. Some things
> > > > aren't compatible.
> > > 
> > > It's called 'less'. Open a bug against it, ask our devs to include
> > > a formatter in 'lesspipe'. Tadaam!
> > 
> > Exactly, now this thread wants to make alternatives to that
> > possible; just because one tool exists doesn't mean everyone wants
> > to use it, there is no one size fits all solution. That's where
> > choice comes from.
> 
> And what benefits do those 'alternatives' give us? Featurism, that's
> all. Implementing new features for the sake of doing something.
> Someone throws a random idea, let's implement it for the sake of
> choice.

Exactly, because an user came up with this; we're not implementing it
yet, we're still discussing it which doesn't mean that there is a team
of people that back certain decisions or implementation choices yet.

> Seriously, how many people actually *care* about
> reading /usr/share/doc with a HTML browser?

That's the question; we don't have statistics on this, maybe we could
make this a question in a future poll.

> How many people actually need it?

Those whom need it, it's not for us to judge what they should use.

> That is, how many people get real
> benefit rather than shiny formatting in their favorite tool.

That's exactly one of the reasons people want to use alternatives.

> Gentoo is not about bending everything upstream provides to match
> every tool a particular user likes.

Actually, it is; "Because of its near-unlimited adaptability, we call
Gentoo a meta-distribution." — http://www.gentoo.org/main/en/about.xml

> Improving the tools give more
> benefit than pushing compatibility cruft.

So, instead of storing it in a format the user appreciates we instead
patch it up in 20 browsers and make maintenance a lot harder?

Or the other option is to implement some kind of conversion HTTP web
server daemon from scratch; it's a lot more work to implement, but
that's the only still reasonable tool I could come up with. And it
doesn't necessarily have to do Markdown to HTML conversion alone; it
could possible be used to yield PDFs on the fly, have an interface you
can use to browse and search /usr/share/doc and so on...

Implementing such daemon wouldn't follow the KISS principle anymore.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] markdown docs like README.md

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

On Wed, 25 Sep 2013 20:15:57 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2013-09-25, o godz. 14:38:14
> Tom Wijsman <TomWij@gentoo.org> napisał(a):
> 
> > On Wed, 25 Sep 2013 08:36:56 +0200
> > Martin Gysel <m.gysel@gmail.com> wrote:
> > 
> > > Am 24.09.2013 19:49, schrieb hasufell:
> > > > I wonder if it would make any sense to take the effort to
> > > > convert markdown docs to html format before installing them.
> > > > 
> > > 
> > > following that logic we should also consider converting all man
> > > and info pages to html...
> > > 
> > > it doesn't make any sense...
> > 
> > Yet there are many websites that show these converted man pages, as
> > well as users looking them up there; that it doesn't make sense for
> > you doesn't necessarily mean that nobody else is doing it.
> 
> *Websites* is the keyword here.

Yes, it connects the dots between man pages and a browser.

> > > simply provide or suggest a markdown
> > > capable viewer to the user
> > 
> > Yet another viewer I have to install; imagine that I had to do the
> > same for video formats, then I'd have to install an AVI player, a
> > MKV player, a MP4 player, a MOV player, an OGG player, a VOB
> > player, ...
> > 
> > No, I'd rather have a single converted format and/or a single
> > player.
> 
> Yet you don't mind most of our users installing a Markdown converter
> (and possibly even more converters) in the sake of converting docs to
> the format you like...

Exactly, it is a single tool as opposed to multiple.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]