Introduction
RSS-torrent is a broadcatching application that watches RSS feeds of Twitter timelines from your favorite torrent sites.
The feed information is passed through used defined filters, that match on title and size of the torrent in order to find the episodes the user wants to download.
The application provides `nodup` filters preventing the download of old or duplicate episodes, so you don't end up with 3 downloads of the same episode.
When a torrent is found that matches a filter and site to the filters you specified the torrent is downloaded and the data is stored to prevent duplicate downloads in the future.
When configured an email can be send to your email address to informyou on the torrent on the way.
RSS-torrent provides for 2 types of filters, the SQL filters and simple filters. In most cases (99%) the simple filters will do just fine for you.
When you really want control on what is downloaded and what's not, please look in to the examples.sh file, and study the SQL filters there.
RSS-torrent does not do the downloading itself, it leaves the handling of the torrentfiles to a specialized Bittorrent program like Rtorrent.
How to configure both programs to work together, can be found in the howto section of this site.
Build this program
BEWARE : When upgrading from before 0.6 to 0.7 or higher, please make sure to backup your filters, because the database will get wiped !
Libraries :
| library | home page |
|---|---|
| Sqlite3 |
http://www.sqlite.org/ |
| Libpcre | http://www.pcre.org/ |
| LibCURL |
http://curl.haxx.se/libcurl/ |
| LibXML2 |
http://xmlsoft.org/ |
| Libesmtp |
http://www.stafford.uklinux.net/libesmtp/ |
Build command:
# cmake .
# make
# make install
Setup RSS-torrent config values.
# rsstorrent --list-config
# rsstorrent --set-config="<item>:<value>" <- have a look at the man page for the settings and there meaning.
The directory to here the torrents are downloaded must exist and be writable to the user running RSS-torrent.
Change the directory using # rsstorrent -C "torrentdir:<path your torrents need to go>"
Or create the directory on the default path # mkdir ~/torrents
To create some example filters and sources in the database
# make examples
For examples look at the examples.sh file
Setting up config variables
List config values :
rsstorrent --list-config
Set a configvalue :
rsstorrent --set-config <name:value>
Example :
./rsstorrent --set-config="refresh:1800"
Config values :
| Value | Description |
|---|---|
| torrentdir | Path the downloaded torrents are placed in. (and the torrent softaware should monitor for torrents) |
| logfile | Path to logfile. |
| lockfile | Path to lockfile. (The lockfile contains the PID op the current running deamon) |
| refresh | Seconds between refreshes. (fetching RSS feeds) |
| default_filter | The default rss filter to add to new rss sources. (defaults to eztv as this is the only working filter atm) |
| smtp_enable | 'Y' is send email notifications on downloading a new torrent, 'N' is don't. |
| smtp_to | Email address to send the notifications to. |
| smtp_from | The from email address the email shows. |
| smtp_host | The STMP server used to send the notifications. |
| retain | The number of days feed information is retained. |
Running RSS-torrent
To start RSS-torrent in daemon mode :
# ./rsstorrent --run
To start RSS-torrent without detaching from the terminal :
# ./rsstorrent --run --nodetach
In order to see if RSS-torrent is still running :
# tail -f ~/.rsstorrent/rsstorrent.log
To stop RSS-torrent
# kill `cat ~/.rsstorrent/lockfile.pid`
Adding sources
Example
rsstorrent --add-source "Eztv:http://www.ezrss.it/feed/" -t "defaultrss" or rsstorrent --add-source "Piratebay:http://rss.thepiratebay.org/205" -t "defaultrss"
The first option -s tells RSS-torrent to expect an <name>:<url> combination.
The first ':' that is found in the string is the delimiter, any ':''s after that are included in the url.
The second option the -t selects what filter is to be used in order to convert the RSS into database records in the 'newtorrents' table.
Adding simple filters
Example
rsstorrent --add-simple='House' --nodup='newer' --title='^house' --exclude='hunters international' --max-size='700.00 MB' --min-size='200.00 MB'
This filter filters for all torrents with the name beginning with "house" and excludes "House hunters international" which we don't want.
To avoid downloading the HD version of the House episodes I added a maximal size of 700Mb, a good quality episode should be at least 200Mb.
The nodup filter states that we only want newer episodes, no reruns for us.
In the man page of RSS-torrent, you can find more information on simple filters.
Fields in the database
The database within RSS-torrent contains 2 tables that are important when filters are created.
The first table is the newtorrents table, this table contains all torrents that passed the filter from source RSSes. The use of the table is to look for new programs to download.
The second table is the downloaded table, all relevant information of the torrent that is being passed to the torrent-program is stored here.
The primary use of the table is to avoid duplicate downloads.
| Table new torrents | |
|---|---|
| id INTEGER PRIMARY KEY | The id of the record |
| title TEXT | The Title of the show |
| link TEXT UNIQUE | The url to the torrent |
| pubdate DATE | The Date the show was released |
| category TEXT | The category the show belongs int |
| season INTEGER | The season number of the show |
| episode INTEGER | The episode number of the show |
| seeds INTEGER DEFAULT 0 | The number of seeds associated with the torrent |
| peers INTEGER DEFAULT 0 | The number of peers associated with the torrent |
| size INTEGER | The size of the torrent in bytes |
| new TEXT DEFAULT 'Y' | This value is 'Y' when first parsed, after that 'N' |
| Table downloaded | |
| id INTEGER PRIMARY KEY | The id of the records |
| title TEXT | The title of the show |
| link TEXT UNIQUE | The url to the torrent |
| pubdate DATE | The date the torrent was released |
| category TEXT | The category the show belongs in |
| season INTEGER | The season number |
| episode INTEGER | The episode number |
| date DATE | The Date the torrent was downloaded |
Adding filters
Filters are used to get the content you want, and avoid duplicate downloads. Filters are made up from 2 parts, the filters part that filters out the candidates for download. And the avoid duplicates part, that looks if a match the filters has already found isn't downloaded before. Both filter parts are sqlite3 queries, you can create them as you like, as long as the following rows are selected for the filter: link, title, pubdate, category, season, episode note: the fields should be selected in this exact order. The duplicate query can select any field it wants because the programs only looks at row count any row count of => 1 indicates a torrent has been downloaded before.
Filters can be tested with the -q option together with the -F and -T option. The query is executed on a sandboxed database, where all 'new' flags are set to 'Y'. After the query has executed, a list of files is printed that would have been downloaded by this rule.
When this list consists of the correct files and no doubles are found, the filter should be okay. By removing the -q option from the commandline, the filter can be inserted. Note: for this feature to work relevant data must be present in the newtorrents table.
Examples
Test filter:
rsstorrent --test --add-SQL-filter "DollHouse:select link, title, pubdate, category, season, episode
from newtorrents where title REGEXP('^[Dd]ollhouse') AND size < '400000000' AND new = 'Y'"
--nodup-sql-filter "SELECT title FROM downloaded WHERE link=?1 OR (season=?2 AND episode=?3 AND title REGEXP('^[Dd]ollhouse'))"
Enter filter:
rsstorrent --add-SQL-filter "DollHouse:select link, title, pubdate, category, season, episode
from newtorrents where title REGEXP('^[Dd]ollhouse') AND size < '400000000' AND new = 'Y'"
--nodup-sql-filter "SELECT title FROM downloaded WHERE link=?1 OR (season=?2 AND episode=?3 AND title REGEXP('^[Dd]ollhouse'))"
The --add-sql-filter option expects "<name>:<filter query="">" the name will be the name of the filter.
The filter query selects the rows that match a query.
the rows that are selected in the query are mandatory, the where clause is free to write your self.
The --nodup-sql-filter argument expects <no duplicate="" query=""> when this query returns 1 or more records, the download candidate is considered a duplicate and ignored.
The selected fields are of no influence here,although 1 row should be selected.
For more information or suggestions to this program, Please contact me at paul [at] interweps.nl
TODO
- Create page with filters for the most popular series
- Twitter support
- Simple filter creator
Contact
For bug reports and feature requests please contact me at rsstorrent [at] swarmtv.nl
SVN
http://svn.etv.cx/listing.php?repname=rss-torrent&
Disclaimer
This program is free software: you can redistribute it and/or modifyit under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, SOUL OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.