Archive for category Programming

Sickbeard with Integrated Torrent Client

Note: Since this post was written, TvTumbler was born. Going forward, all new development will be in TvTumbler.


Sickbeard Integrated Torrent ClientI’ve included a torrent client in my torrent-friendly fork of Sickbeard. The UI is a little limited at the moment, but it’s been wiping the floor with both Transmission and uTorrent here for the last few days in terms of download speed, so I think it’s ready to be set loose on the world.

To enable it, you’ll first need to install libtorrent rasterbar (version 0.16 or later), and its python bindings.

If you do not have an already-running copy of this fork, just follow the general installation instructions.

Otherwise:

For debian derivatives: (i.e. Ubuntu, mint, etc)
sudo apt-get install python-libtorrent

That could hardly be much simpler now could it?

For OSX:
You’ll need to do a little compiling I’m afraid.

This post includes all the required details.

For Windows: (yes, for once I haven’t forgotten about you!)
Go here, and download the .msi which matches your version of python (probably either py27 or py26).
Pick a recent version – the older ones won’t work with magnet links.

Another OS:?
If your OS doesn’t come with a packaged version of libtorrent rasterbar and its python bindings (and note that most will!), then you’ll probably need to build from source.
svn co https://libtorrent.svn.sourceforge.net/svnroot/libtorrent/trunk/ lt/
cd lt/
./autotool.sh
./configure --enable-python-binding
make
sudo make install
sudo ldconfig

Once you have installed the above, restart Sickbeard, go to Config -> Search Settings -> Torrent Method, and change it to “Integrated”.

Any OS-specific tweaks you wish to share? Please note them in the comments.

Notes:

  • Is libtorrent installed? What version do I have?
    python -c "import libtorrent as lt; print lt.version" will tell you very quickly. You should get a response like 0.16.9.0 – if you get an error, or a version less than 0.16, this is likely to be your issue.
  • All my torrents are stuck at “downloading_metadata”: for dht to work (and hence for metadata to be available), the torrent client needs at least one valid peer to exchange data with, but if all your downloads are trackerless magnets, it won’t have one. Try adding a torrent manually (using either a magnet link with trackers, or a link to a .torrent file). You can delete the added torrent as soon as metadata begins to download. [Note: This issue should be resolved with recent releases]
  • Having other Issues? Check out this post (and read through the comments) to see if someone else has addressed it.
  • Still having Issues? Check out the github issue tracker. Please search the existing issues before creating a new one, and include a debug log in any new issues! (you can censor it beforehand if you feel the need)
  • It looks like ubuntu 12.04 installs libtorrent 0.15.x by default, which is a little too outdated for use here. So, if you are a 12.04 user, you’ll either need to build from source, or find a ppa with a more recent version (please post any suggestions in the comments). Thanks go to Chris for pointing this out.Update (with thanks to Dennis): This repo appears to have all the needed packages.

, , ,

50 Comments

Building Libtorrent-rasterbar on OSX

Compiling Rasterbar Libtorrent on OSX 10.7 or 10.8 (tested on both 10.7.5 and 10.8.4), without brew and all the crap it brings with it.

First, you’ll need to install XCode, and command-line tools.
Then, you’ll need to download and build boost:

http://www.boost.org/users/download/

Click on ‘Download’, and download the latest .tar.gz. In my case this was boost_1_53_0.tar.gz

tar -zxvf boost_*.tar.gz
cd boost_*
./bootstrap.sh
./b2
sudo ./b2 install

This all ran cleanly in my case, but googling would seem to imply that it often gives errors.
Next, download the libtorrent source:

http://code.google.com/p/libtorrent/downloads/list

And again, download the latest .tar.gz. In my case this was libtorrent-rasterbar-0.16.9.tar.gz

tar -zxvf libtorrent-rasterbar-0.16.9.tar.gz
cd libtorrent-rasterbar-0.16.9
./configure --enable-python-binding
make
sudo make install

That should be it, but in my case for some reason the python bindings were put into /usr/local/lib/python2.6/site-packages/ (where the correct site-packages is in fact /Library/Python/2.6/site-packages). I’m not really sure why this happened, but it’s easy enough to fix. Continuing from where we were above:
cd bindings/python
sudo python setup.py install

And it gets the correct directory this time around.

Update (Sept 2, 2013): Tested with libtorrent 0.16.11 and boost 1.54 on Max OSX 10.9. Works as described above. Boost gives lots of deprecation warnings, but it runs without issue.

,

4 Comments

Sickbeard Manual RSS/Custom Torrent Providers

Note: Since this post was written, TvTumbler was born. Going forward, all new development will be in TvTumbler.


Custom RSS Torrent feedI’ve added experimental support for Custom RSS torrent feeds into my sickbeard fork.

The whole thing should be self-explanatory I think. Currently I use the following feeds myself:

  • TPB – TvTeam: http://rss.thepiratebay.sx/user/7e434ea6455f089db7f0291b3bd1d4db
  • TPB – VTV: http://rss.thepiratebay.sx/user/e395935fa39597e3997b916a295f23e4
  • TPB – ettv: http://rss.thepiratebay.sx/user/0e3b2f46e5892dc5b2aac6dbe548bc27

These are links to the rss feeds for specific users on thepiratebay.se.
Obviously any rss feed that includes links to download torrent files (or public magnet links) will work also.

Other feeds that have been recommended to me (but I don’t use myself) include:

  • BT-chat: http://rss.bt-chat.com/?cat=9
  • monova.org: http://www.monova.org/rss.php?type=catname&id=8

(Please feel free to recommend others in the comments)

Consider this experimental for now. It works for me, but I can’t guarantee that it will work for anyone else.

Check out this post for more details about this fork.

,

33 Comments

Netatmo Menu Bar app for OSX

Netatmo Menu Bar ScreenshotFor some peculiar reason, Netatmo don’t have an OSX app for viewing their devices (however their iOS one is simply brilliant). So, me being me, I wrote one. Very simple app to put together in fact (but then, it doesn’t do all the fancy history stuff that the iOS one does).
Read the rest of this entry »

,

8 Comments

Sickbeard + iPlayer: requirements

Note: Since this post was written, TvTumbler was born. Going forward, all new development will be in TvTumbler.


BBC iPlayer LogoI’ve just merged the iplayer branch of my Sickbeard fork into master, as it’s been working ok for me for the last few days, and I think it’s ready for some others to check it out.

But, be warned: it has prerequisites. In short, these are:

  • Perl: any *nix will have this
  • ffmpeg:
    • On linux, you probably just need to do sudo apt-get install ffmpeg (or whatever the rpm equivalent is).
    • On OSX, take a look at my recent post about how to build ffmpeg.
  • rtmpdump:
    • On linux, again, you can probably just do sudo apt-get install rtmpdump (or the rpm equivalent).
    • On OSX, building this is just too simple:
      git clone git://git.ffmpeg.org/rtmpdump
      cd rtmpdump
      make SYS=darwin
      sudo make install SYS=darwin
  • If you’re on a *nix machine, there’s a strong likelihood that you have all these installed already, so best to check first.

    Windows users: you’re on your own here, sorry. I believe that get_iplayer will run on Windows, but I simply don’t have a copy of the OS installed on which to check it. You’ll probably need to get both ffmpeg and rtmpdump also too I’m afraid. If you do have success with windows (or indeed any other OS), let me know in the comments and I’ll add the instructions here.

    All of this is made possible of course by the work from infradead on the get_iplayer script (latest version included in the sickbeard code, but you can install your own if you prefer).

    Please note: for this to work you must either be *in* the UK, or have some kind of VPN connection to the UK. iPlayer downloads are restricted to UK ip addresses. (of course there are other ways around this, but I’m not going to discuss those here).

,

3 Comments

Building ffmpeg on OSX 10.7

Note-to-self post again.

I can take no credit for this – it’s a copy of Hunter Ford’s script from here, with some minor changes I had to make to get it to run cleanly on 10.7.5.

Prerequisite: XCode, with command line tools installed.

Read the rest of this entry »

, ,

1 Comment

MySQL: moving innodb tables out of ibdata1

Note-to-self style post again.

MySQL’s innodb will store all table data in one file called ibdata1, unless you tell it to do otherwise. This can become an issue sometimes, particularly when the file gets really big.

Thankfully there’s a way to fix it.

Add innodb_file_per_table to my.cnf

[mysqld]
innodb_file_per_table

Update the running server with the same value
SET GLOBAL innodb_file_per_table=1;

Then
ALTER TABLE tablename ENGINE=Innodb;
for each table to move it out of the ibdata1 file.

Alternatively, you can use this little bit of trickery to generate the sql needed to do it for all tables:

SELECT concat('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME, ' ENGINE=Innodb;') FROM INFORMATION_SCHEMA.TABLES WHERE ENGINE = 'InnoDB';

Then paste the result as sql and run it (it may take some time though).

,

No Comments

Building goaccess on OSX 10.8.2

Title says it all really. Copy/paste of what was needed to build goaccess on 10.8.2, without pissing around with brew et al.


tar -zxvf goaccess-0.5.tar.gz
cd goaccess-0.5

curl -L http://ftp.gnu.org/pub/gnu/gettext/gettext-0.18.1.1.tar.gz -o gettext-0.18.1.1.tar.gz
tar -xf gettext-0.18.1.1.tar.gz
cd gettext-0.18.1.1
./configure CC="gcc -arch i386 -arch x86_64" CXX="g++ -arch i386 -arch x86_64" CPP="gcc -E" CXXCPP="g++ -E"
echo '--- gettext-tools/gnulib-lib/stpncpy.c.orig 2007-10-07 23:29:35.000000000 +0300' > gettext.patch
echo '+++ gettext-tools/gnulib-lib/stpncpy.c 2011-03-11 23:34:40.000000000 +0200' >> gettext.patch
echo '@@ -24,7 +24,7 @@' >> gettext.patch
echo ' #include ' >> gettext.patch
echo ' ' >> gettext.patch
echo ' #ifndef weak_alias' >> gettext.patch
echo ' -# define __stpncpy stpncpy' >> gettext.patch
echo ' +//# define __stpncpy stpncpy' >> gettext.patch
echo ' #endif' >> gettext.patch
echo ' ' >> gettext.patch
echo ' /* Copy no more than N bytes of SRC to DST, returning a pointer past the' >> gettext.patch
patch -p0 < gettext.patch
make -j2
sudo make install

curl -L http://ftp.gnome.org/pub/gnome/sources/glib/2.27/glib-2.27.0.tar.bz2 -o glib-2.27.0.tar.bz2
tar -xf glib-2.27.0.tar.bz2
cd glib-2.27.0
./configure --with-libiconv=native

echo '--- glib-2.27.0-orig/glib/gconvert.c2010-09-17 17:33:50.000000000 -0500' > glib.patch
echo '+++ glib-2.27.0/glib/gconvert.c 2010-10-27 00:01:39.000000000 -0500' >> glib.patch
echo '@@ -61,9 +61,6 @@' >> glib.patch
echo ' #if defined(USE_LIBICONV_GNU) && !defined (_LIBICONV_H)' >> glib.patch
echo ' #error GNU libiconv in use but included iconv.h not from libiconv' >> glib.patch
echo ' #endif' >> glib.patch
echo '-#if !defined(USE_LIBICONV_GNU) && defined (_LIBICONV_H)' >> glib.patch
echo '-#error GNU libiconv not in use but included iconv.h is from libiconv' >> glib.patch
echo '-#endif' >> glib.patch
echo ' ' >> glib.patch
echo ' /**' >> glib.patch
echo ' * SECTION:conversions' >> glib.patch
make
sudo make install

cd ..
./configure
make
sudo make install

,

2 Comments

Sickbeard Fork: Feature Summary

Note: Since this post was written, TvTumbler was born. Going forward, all new development will be in TvTumbler.


Well I’ve been working on and off on Sickbeard for the last few weeks, and the list of additions is starting to grow. So it’s probably time for a summary of the main additions.

Sickbeard Integrated Torrent ClientIntegrated Torrent Client

Uses libtorrent-rasterbar to provide a very simple (but very fast!) integrated torrent client. Manages queueing, priority, seeding, and deleting of torrent automatically. No need to use blackhole or watch dirs (but of course you can use those if you wish). If libtorrent rasterbar is not available on your system, this feature simply won’t show. You can continue to use your existing torrent client without issue.

Custom RSS Torrent feedCustom Torrent RSS Providers

Add one or more custom torrent providers. Any torrent source with an RSS feed should work for this.
See this post for more info (including some recommended feeds).

 

New Torrent Search Providers

I’ve added all the useful/reliable torrent providers I know as search options (at least the ones that you don’t need invites/accounts for). These are:

  • ShowRSS: This appears to be very reliable, but their show naming is not always smart, and their backlog can be limited.
  • Kickass Torrents: Includes solid backlog support.
  • DailyTvTorrents: Again, appears reliable, and includes solid backlog support (thanks to lylej for the heads-up on this one).(sadly, this is now gone)
  • BBC iPlayer support (via the perl get_iplayer script): See here for instructions on how to set this up.
  • Support for magnet links.
  • PublicHD support: No backlog yet, their RSS is *extremely* limited.
  • And additionally, EZRSS has been “fixed” so that it will continue to work (by using the twitter feed, tpb feed, and a feedburner feed as backups) even when the site goes down (which happens a lot!)

 

Custom Scene Names

The edit show page now includes a box into which you can type ‘custom’ show names (or ‘Scene Names’). These will be searched for in addition to the tvdb show name. This is incredibly useful for picking up on minor variations in show names.

 

Custom Scene Numbering

Anyone who has used sickbeard for any length will know how annoying this is: The scene decides that the first episode was a double, but tvdb (from which sickbeard gets its numbering) thinks otherwise. Or they include a special in the numbering. Or they disagree on season numbers. Or they call the “special” episode “SxE00″, where tvdb calls it “S00E56″.

Well this gives you the facility to work around this problem, without having to manually “fix” everything sickbeard downloads (or tolerate the duplicates it creates). You can now manually enter a season/episode number for each episode, and this is what sickbeard will search for when trawling the internet. And it will renumber it correctly once it’s downloaded to your PVR.

The code will also regularly check thexem.de for the known variations between scene and tvdb numbering – so in most cases you probably won’t even need to correct the episode numbers, sickbeard will do so itself (but of course it will respect your numbering if you have specified it manually).

 


Installation Instructions

These now have a wiki page of their own. See How to Install Sickbeard for Torrents on github.


And I do believe that this time, I’m really finished. These are the main things that have annoyed me about sickbeard since I’ve started using it, and it took far more work than it should have to actually fix them (all-in, I’ve probably spent a full day on this). If I feel tempted to add features to this in the future, I think I’ll start by doing a re-write: it will save time in the long run (and imo sickbeard is badly in need of a rewrite).

As always, the full source is available from https://github.com/bricky/Sick-Beard.

 


Update/Please Note: I don’t intend to add any more features to this – so please stop asking for them! Sickbeard is, as far as I’m concerned, broken – and I don’t have the time to fix it, or write a replacement.
I will generally keep this up to date however, until I find a better system (or actually do the rewrite).

(if someone else wishes to start a rewrite however, please let me know – I may be able to contribute)


Update (Sept 2013): I’m doing the rewrite: TvTumbler, an addon for XBMC with the same basic functionality. I’d encourage any new users (who also use XMBC) to try TvTumbler instead.


Having Issues?: Please use the github issue tracker (and include a debug log, or your issue will likely be ignored)

, , , , , , ,

147 Comments

Air Video addon for XBMC

Air Video logoThis is simple enough really: an addon for XBMC that allows you to browse (and stream) from an Air Video server.

Download Air Video Plugin for XBMC

To install this in XBMC, download the zip file above, save it somewhere you can find it, then

System -> Settings -> Addons -> Install from zip file

and browse for the file you just downloaded
Be sure to configure the “Location” in the preferences before use (this should be the ip address of your air video server).

I can take very little credit for this. Most of the Air Video code comes from Jeremy Debate’s port of JP Hastings-Spital’s ruby library for interacting with Air Video, with some minor modifications here and there to get it to work from within XBMC.

Note that I’ve only tested this on Frodo, the latest development build. I would expect it to work without issue in Eden also however.

UPDATE: Please note that if you open a video by simply selecting/clicking it, it will not be transcoded (xbmc will simply attempt to play the file as-is). Transcoding requires opening the file via the context menu/right-click/’C’-key, and does not appear to work correctly in current versions of XBMC due to lack of support for the stream format. Please read the comments below for further details.
(Thanks for signorRossi for this information).

,

42 Comments

Sickbeard support for ShowRSS & Kickass Torrents


This fork has moved on a bit since this was posted. See here for details.

Sickbeard is a simply wonderful piece of software, but its only real torrent support is for ezrss, which is often (and currently) broken. There’s no real technical reason to restrict it to just this feed (at least none that I could see), so I’ve forked it and added support for another feed called showRSS.

https://github.com/bricky/Sick-Beard

That is all.

Update: Now also includes the ability to add your own scene exceptions through the web UI, and uses twitter rss as a backup for ezrss when it fails.

Update 2: Now also includes support for kat.ph (aka Kickass Torrents), which provides a much wider range of shows (and also, apparently, backlog support).

, , ,

12 Comments

Excel function to calculate numerology number

Function below to calculate the numerology number (reduced to 2 digits) for a name in Excel:

=SUM( 1* CHOOSE( CODE( MID( LOWER(A1); ROW( INDIRECT("1:" & LEN(LOWER(A1)))); 1))-96; 1; 2; 3; 4; 5; 6; 7; 8; 9; 1; 2; 3; 4; 5; 6; 7; 8; 9; 1; 2; 3; 4; 5; 6; 7; 8))

Press Ctrl+Shift+Return to have Excel treat it as an array function (it will enclose it in curly braces { and }).  A1 is the cell reference containing the name.

6 Comments

css.php