Advisories

13 August 2013 - A Note to All Virtual Machine Users

This advisory rectifies issues reflected in the (2) advisories from 7 August and includes important information about some 3rd party dependencies that have been upgraded.

As of 13 August at approximately 11pm (U.S. Central Time, GMT-5) the following changes were pushed to GitHub:

• The Vagrant Berkshelf plugin that was previously required was removed, which simplifies VM installation. Installation of the virtual machine is now one step simpler for everyone since there is no longer a need to install the Vagrant Berkshelf dependency. Mac OS X users no longer have to worry about installing gcc and developer tools, which was never intended to be the case in the first place.
• Users who are working on 32-bit systems with the precise32 image should no longer be affected by the 7 August advisory now that a newer version of jpype (hosted as JPype1 at PyPI) is being used.
• IPython Notebook was upgraded to v1.0.0, which provides a better UX and many enhancements
• The python-boilerpipe and jpype dependencies were updated to be installed from PyPI instead of from forked GitHub repositories, which simplifies behind the scenes configuration management. (Thanks especially to @marsam, @misja, and @originell for helping to make this happen. In the configuration for the virtual machine, it's now as simple as pip install boilerpipe to install this package and it's jpype dependency.)
• A couple of other minor updates to IPython Notebooks also took place involving code tweaks

In order to take advantage of these changes, existing virtual machine users are encouraged to perform the following steps:

• vagrant destroy the existing virtual machine
• git pull to update the working code repository with the latest changes
• vagrant up to bootstrap the new virtual machine

I realize that asking you to vagrant destroy and vagrant up is annoying, and my hope is that this is the last major update to the virtual machine before Mining the Social Web, 2nd Ed. is published. It was important to get in these enhancements to make the virtual machine the best possible experience for everyone. Thanks for your patience and understanding as the code approaches its official launch in tandem with the official release of the book.

As always, please log an issue here on GitHub or reach out via Twitter/Facebook with any questions or concerns.

7 August 2013 - A Note to Users with 32-bit systems

This advisory was obsoleted by the 13 August 2013 advisory

It appears that the precise32 image that users with 32-bit systems should use for their Vagrant base image installs Java in a somewhat unexpected location, so the installation of jpype (one of the 3rd party dependencies that is handled by the virtual machine) currently fails.

The workaround is rather simple, but needs to be tested before it is rolled out so that no regressions occur. If you are a user with a 32-bit system and need support before a fix is in place (estimated turnaround time on the fix is not later than 11 August), please reach out on Twitter, Facebook, or here on GitHub.

Hopefully, this affects a relatively small number of users. A update will be added to this advisory reflecting that it is no longer an issue once the problem is fixed in the GitHub repository.

7 August 2013 - A Note to Mac OS X Virtual Machine Users

This advisory was obsoleted by the 13 August 2013 advisory

In order for Mac OS X users to complete the vagrant plugin install vagrant-berkshelf step of the setup process, it turns out you need to have a developer tool that compiles C-based source code called gcc on your machine. The reason is because vagrant-berkshelf has dependencies that require C code to be compiled behind the scenes.

If you are a developer and have XCode or Homebrew installed, there is a good chance that you may already have gcc on your machine and need to do nothing, or you might be technically inclined to go about acquiring it in your own preferred way. However, if you use a Mac and are not a developer and haven't installed developer tools such as the command line tools that come with XCode or a gcc toolchain from Homebrew, you will experience an error in trying to install vagrant-berkshelf, which prevents you from ever completing a successful vagrant up.

Fortunately, the fix is simple, although it does involve the one extra step of getting gcc on your machine. By far, the simplest route is by installing it from the osx-gcc-installer. Just download the package file, follow along with the wizard, and then complete the installation of your vagrant-berkshelf plugin. That's all.

However, be sure that don't already have gcc installed before installing this additional package. You'll know that you don't have it because your attempt to install the plugin will have failed, and you'll have gotten directed to this advisory, or because you'll have proactively typed gcc in a terminal before attempting any installation and get a result back to the effect of "gcc: fatal error: no input files", which indicates that you do have gcc on your system

Other options for getting the developer tools involve a little bit of work that you can read about here.

This advisory does not affect Linux or Windows users. If you are a Linux developer, then it is almost guaranteed that you have gcc (or know how to get it), and it turns out that Vagrant for Windows ships with an embedded version of gcc.

An annotation is added to the YouTube video and the Appendix A instructions are updated to reflect this advisory.

As always, reach out on Twitter, Facebook, or here on GitHub if you need help.

26 July 2013 - An Important Note to All Virtual Machine Users

A significant pull request was merged in on July 26, 2013 around 9pm (CST) that provides an enhancement to the way the book's Vagrant-based virtual machine is configured. If you were an adopter of the book's virtual machine on or before 26 July 2013, you will probably want to do complete the following steps to get in a good spot for moving forward with the code. If you pulled the code after July 26, this advisory will not affect you.

• Save any work in IPython Notebook that you don't want to lose by copying the ipynb file to another location (or by using git stash)
• vagrant destroy - Kill the existing virtual machine
• git pull - Update your repository's code
• vagrant plugin install vagrant-berkshelf - Install a required Vagrant plugin
• vagrant up - Re-bootstrap your virtual machine with Chef-based configuration management. The first bootstrap takes ~20 minutes, which is significantly faster than the previous approach.

Update on 29 July: Thanks to David Rush (@DDucks) for pointing out that you should do the vagrant destory before doing the git pull.

Enjoy!

28 July 2013 - Added large file to GitHub repository

In completing a final review of the manuscript and doing another round of testing on the code, I realized that the time that it takes for Vagrant to perform synchronization of the thousands of files that are unarchived from the Enron corpus can take a very long time -- potentially upwards of 2 hours on some systems. Although preprocessing the original data is a very worthwhile exercise, a 2 hour delay is far from acceptable for users who are taking advantage of the virtual machine to streamline learning.

As a reasonable workaround, I checked in a highly compressed version of the output from Example 6.3 so that readers who are either not interested in the details of preprocessing the original corpus or just don't have the time to wait can proceed throughout the real substance of the chapter without additional delays. The IPython Notebook for Chapter 6 has been updated with notes that clearly explain the steps involved.

Bottom line: You can opt to downloading the original Enron corpus and you won't need to execute Examples 6.2 or 6.3 if you'd like to opt-out of the potentially time consuming preprocessing.

Enjoy!