notebooks not building into the book

[darribas]

Hi,

I've clone this repo, got the gitbook setup and built it but the website only pulls in the markdown files, not the notebooks. Am I missing something or the book doesn't currently include the notebooks?

Thanks very much!

[patricksnape]

Hi Dani,

Thanks for checking out the notebooks! We are currently in the midst of a transition into how we are going to serve/present the website and the notebooks content. If you simply want to use the notebooks then you don't need to the gitbook at all. In fact, it's very very unlikely you would want the gitbook at the moment as we are still in the middle of working out how we want to use it!

So just clone the repo and point your Jupyter instance at the notebooks folder and you should be able to run through the notebooks. Soon the website will update and this will be explained a lot more clearly!

Cheers!

[darribas]

Thanks very much for your answer. I actually found the repo as the only example I could find of a gitbook project sourcing notebooks, and that's what I was trying to figure out. Did you manage to include them in the book project? Sorry for asking a marginally relevant question to the menpo project, but it'd be of great help for me.

[patricksnape]

Oh I see! Sorry! @nontas @jabooth you are the gitbook experts - do we/did we get notebook sourcing working?

[darribas]

Thank you very much @patricksnape, it'd be fantastic to get this working!

[nontas]

Hi @darribas! We don't build the gitbook based on notebooks (if I understand your question correctly). We write all the markdown files from scratch. Unfortunately, I haven't ever tried building a gitbook sourcing from notebooks.

[jabooth]

hey @darribas!

So the first comment is that we have a far more updated gitbook over at menpo/book, this was just my early testing and proof of concept which @nontas has really ran with since.

We haven't yet tackled integrating the notebooks into the gitbook yet, but it's definitely something we are looking to do. As I see it, there are two options, one easy but very limited, one difficult, fragile, but far more elegant from a UX perspective:

1. Manually have an iframe on the book which points to the github repo/nbviewer for the notebooks, e.g.
   http://nbviewer.jupyter.org/github/menpo/menpo-notebooks/tree/master/notebooks/
   This will be grim in a few ways - navigation through books is completely separate from the gitbook, and obviously the style will not match at all, but for us it's a pretty efficient way to move towards notebook integration.
2. Run nbconvert on all notebooks, generating HTML for each, then integrate these into the gitbook. Rough plan here would be to develop a notebook_template.md page that included templates for where (HTML) notebook content would be injected. We could customise the CSS around the notebook fairly straightforwardly, and make a notebook theme that matches the gitbook theme. We'd also want the notebooks to be in the contents page somewhere, so some form of tempting into the contents page would be required.

I think the second approach is the way to go long term, but it could be a little bit of an undertaking. I might have a hack on it sometime and share back on here if I have any success. I think there is probably broader interest in this though, so it might be worth spinning out any notebook/gitbook integration code into it's own little project that a community can work on (I could definitely imagine future notebooks breaking existing formatting etc, so it would be nice to share the burden of keeping up to date with changes).

How were you thinking of tacking this @darribas? Would love to collaborate on a solution if it's something you are keen to push forward with!

[darribas]

Thanks very much for the answers! @jabooth my original idea was along your option 2. I've pinged the gitbook folks to see if there's any obvious thing I'm missing here but no response so far. My intuition was that the easiest way would be to convert the notebooks into stripped down html (using nbconvert --to html --template basic) and then feed that into gitbook. I run into the problem that gitbook will only accept markdown apparently, even though the final output is going to be html, so I'd imagine there's no real problem under the hood. I think if we managed to get gitbook to simply take an html file listed in SUMMARY.md and feed it into the book directly (without going through the markdown parser), that'd do the trick just fine. The rest I think is a bit of eye-candy in terms of syntax highlighting and so on, but I think most of it is directly supported by gitbook anyway.

I'm keen to get this to work, and I agree more people would be interested in this. I don't have much experience in rolling out plug-ins or packages and none in JavaScript so not sure I could contribute much in the form of a full-fledge package, but definitely interested in getting it to work (even if stlightly rough/manual in the beginning) and carefully documenting it so others more versed in the packaging can do that part.

[jabooth]

My intuition was that the easiest way would be to convert the notebooks into stripped down html (using nbconvert --to html --template basic) and then feed that into gitbook.

Ah that's awesome, didn't realise there was a --template basic option! Yup, this sounds good

I run into the problem that gitbook will only accept markdown apparently, even though the final output is going to be html

That's true, but you are free to mix HTML markup in the markdown files.

For instance, the top of our new homepage uses some HTML at the top in order to style a centralised image:

<center>
  <img src="logo/menpoproject_white_medium.png" alt="The Menpo Project" width="30%"></br>
  <strong style="font-size: 250%">The Menpo Project</strong>
</center>

I'm not sure what the limitations on this are, but my feeling would be that's much easier to develop a solution for this that is simply written in Python and outputs valid markdown files for gitbook to process, without requiring any change at all on gitbooks part. I also feel that the template markdown file may well contain more stuff than simply the output from nbconvert - for instance, you could have at the top of the template a heading which includes a custom Download notebook button.

[darribas]

OK I've just tried with a notebook example and the results are encouraging but not perfect yet.

If I do:

jupyter nbconvert --to html --template basic mynb.ipynb
mv mynb.html mynb.md

and plug mynb.md into gitbook, it looks pretty good with too main exceptions:

1. No syntax highlighting.
2. After many cells, there is a few </div> that get printed. For example, for the beginning of this notebook, the following gets rendered by gitbook:

Other than that, it looks awesome and super simple. Any ideas @jabooth ?

[jabooth]

@darribas awesome! Any chance you can put a demo of this live somewhere so I can take a look at the HTML? Few thoughts:

1. The HTML output by nbvconvert will be I imagine a full HTML document with e.g. <body> tags in there - these may be causing an issue if they are being placed in the .md file which could lead to some misalignment of tags (I could imagine that leaving those </div> in the document for example). Can you put the HTML generated by your command in a gist so I can take a look?
2. I'm guessing syntax highlighting is just not enabled in the basic template, is that the case? Again, if we can compare the HTML direct from nbconvert with the HTML after being processed by gitbook, I'm sure we can get to the bottom of all these issues.

This does look really promising though for a first bash though! Does the open street map widget come though OK?! Based on this it seems like we might be able to get a really nice solution pretty quicky 👍

[darribas]

I've just pushed my experiments into a repo:

https://github.com/darribas/test_book

chapter-1/sub3.md is the convert of lab_03.ipynb with a basic template.

Let me know if you make any progress, super keen on getting this done :) It'd amazing if we could iron out those small kinks!!!

[patricksnape]

Don't convert to HTML - convert it to markdown using nbconvert: jupyter nbconvert --to markdown blah.ipynb --output blah.md.

I just tried it on your repo and it looks much better and the images actually work correctly and code is syntax highlighted.

[jabooth]

Whoops, didn't realise markdown was a valid target! That's nice. Does the notebook look themed then, and fit in with gitbook @patricksnape? If so job done!

[patricksnape]

There's not theming at all, it's just markdown!

[darribas]

I vaguely remember trying that out yesterday and not fully working, but I might have done something wrong... At any rate, markdown does imply simplification so some things might get lost. It should however pick up python...