WebGrab+Plus and Argus - Supporter Thread

Discuss XMLTV grabbers or get help with XMLTV grabbing here
Malyngo
Posts: 70
Joined: Wed Sep 08, 2010 8:25 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Malyngo » Fri Aug 22, 2014 12:24 am

Christoph21x wrote: EDIT: PS - do you know a good site with relative numbering? If so, we could create an ini for that one...
Ah! Just found out tvtv.de has relative numbering!
The provided ini ends up with this information in the xmltv:
<episode-num>Staffel 7, Folge 17 von 24</episode-num>
So I think this should be pretty easy to change the ini to get something like S07E17. This would be exactly what I need!
It also has the subtitle of the episode in it.
(Checked this on Sat1 with "Criminal Minds", but I guess it should be the same for other channels)

Edit: There is also a ini with xmltv_ns style:
<episode-num system="xmltv_ns">6.16/24.</episode-num>
If I am not mistaken, this should probably work out of the box. Will check tomorrow evening, it's really late now.

Christoph21x
Posts: 1070
Joined: Wed Oct 19, 2011 9:39 am
Location: Berne / Switzerland

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Christoph21x » Sat Aug 23, 2014 8:24 am

Hi Malyngo - did you test it? Would be interested, too, in a working relative numbering guide in German.
Thanks in advance & greetz - Chris
"One must still have chaos within oneself, to give birth to a dancing star." (F. Nietzsche)

Malyngo
Posts: 70
Joined: Wed Sep 08, 2010 8:25 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Malyngo » Sat Aug 23, 2014 12:13 pm

Yes, works like a charm.
It will also work with the xmltv_ns version you can download from the Webgrab website, but I also created a version with onscreen style, like S02E04.
https://onedrive.live.com/redir?resid=E ... file%2cini
This version does it in the style S02E04/13 if the number of episodes in the season ist also available on the website, but you can easily change that if you want the plain S02E04 in there.

If you want to make a schedule that should just record season 2, you can create that by "S02" into the Episodenumber field with "starts with".

Really happy, now also my recordings on stupid Sat1 and kabel eins have the correct Season information. The GuideEnricher just couldn't match those, as Sat1 would insist on NOT having any matchable information in their DVB EPG stream.

raki
Posts: 24
Joined: Fri Oct 04, 2013 9:33 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by raki » Mon Aug 25, 2014 4:49 pm

Hi Malyngo,

that sound great! Could you give us a short summary / steps of installing and setup WebGrab+ like you did?

Thanks!

Malyngo
Posts: 70
Joined: Wed Sep 08, 2010 8:25 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Malyngo » Mon Aug 25, 2014 5:44 pm

This is pretty straight forward.
I just followed the install and configuration instructions on the WG++ site:
http://webgrabplus.com/documentation

You can also check if there is a newer site.ini on their page:
http://webgrabplus.com/epg-channels

If you download a site.ini package there, it will consist of the ini itself, and a channels.xml.
The ini you just copy to the WG++ directory (consult the documentation for that), and the channels.xml you need as a lookup to see what goes into the WG++ config.

They even have instructions how to integrate with ArgusTV and run it in a Windows schedule:
http://webgrabplus.com/documentation/in ... e/argus-tv

I currently don't even have the automatic download, still running WG++ manually, and then run a bat file to copy it over to the XMLTV folder of ArgusTV, where it will be processed automatically by Argus.

I recommend you start with a timespan in the Webgrab config of 0, to only download the current day, because this will finish quite fast. Even just put a few channels in there, not all of them.
So you can check if the data you have in the guide is what you want, because in the beginning you might want to check different sites to scrub.

I chose tvtv.de because they have relative Season/episode numbers, but tvtoday for example might have the better guide data for you, because they also have ratings. So it depends on your preferences which site ini is the best one for you to grab your data.

But what you can also do is to grab data for some channels from one site, and a few other channels from another. It is really flexible.

Edit:
Ah, yeah, what I just remembered: It can be a bit of a hassle if the channels in Argus are named differently than what you get from the site you are grabbing from. What I did was to make sure in the WG++ ini to name them exactly like in Argus. So you don't have to do anything in Argus, maybe except of setting to use XMLTV over DVB EPG.
In my case, I am using the HD channels, but the EPG is grabbed for the SD ones.
So, if you are taking the configuration for your channels from the channels.xml for your site, there is something like this in there:
<channel site="tvtv.de.xmltv_ns" xmltv_id="Sat.1" site_id="SAT1" update="i">Sat.1</channel>
But my channel in Argus is called: Sat.1 HD
So I changed that to
<channel site="tvtv.de.xmltv_ns" xmltv_id="Sat.1" site_id="SAT1" update="i">Sat.1 HD</channel>
and Argus will know to import that EPG data to the right channel.

raki
Posts: 24
Joined: Fri Oct 04, 2013 9:33 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by raki » Mon Aug 25, 2014 7:46 pm

Thanks for your fast answer! Could you provide your setting / files? Tomorrow I will configure my htpc and a working reference would be cool :).

raki
Posts: 24
Joined: Fri Oct 04, 2013 9:33 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by raki » Tue Aug 26, 2014 8:32 am

Hi,

well, tvtoday seem to work, tvtv.de apparently not...Log files says the following:

Code: Select all

WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/18 -- Jan van Straaten
-----------------------------------------------------------
Job started at 26.08.2014 08:30:19

element site: System.ArgumentOutOfRangeException: Der Index und die Länge müssen sich auf eine Position in der Zeichenfolge beziehen.
Parametername: length
   bei System.String.Substring(Int32 startIndex, Int32 length)
   bei WebGrab.SiteIni..ctor(String ini_file, Utils u)

Input file C:\ProgramData\ServerCare\WebGrab\guide\guide.xml not found   ... created a new one ... 

No EPG update requested!

Starting MDB Postprocess

LocalMDB file C:\ProgramData\ServerCare\WebGrab\mdb\mdb.xml not found   ... created a new one ... 
XMLTV file - C:\ProgramData\ServerCare\WebGrab\guide\guide.xml - doesn't exist
Check filename and location!
Cannot run MDB postprocessor


MDB Postprocessor result :

Mdb data found for 0 out of 0 Movies-candidates
Mdb data found for 0 out of 0 Series-candidates
In 00:00:00.0210002 seconds


Finished Postprocess MDB
Any idea?


Edit: I'm not quite sure what exactly is editable with the ini file. For example, with tvtoday I got the following infos (see attachment). Is it possible to put the Season and Episode information in the detail windows as well? What else can be modified?

Thanks!


Edit2: I thought I might have forgotten to install the upgrade of WebGrabber+. After I did this, the log says the following:

Code: Select all

WebGrab+Plus/w MDB & REX Postprocess -- version 1.54.6/0.01 -- Jan van Straaten
-------------------------------------------------------------------------------
job started at 26.08.2014 09:17:06


reading config file: C:\ProgramData\ServerCare\WebGrab\WebGrab++.config.xml
loading timezone data
embedded timezones source: Webgrab_Plus.TimezonesData.txt
found: C:\ProgramData\ServerCare\WebGrab\tvtv.de.ini -- Revision 8

running on Microsoft Windows NT 6.2.9200.0

input file C:\ProgramData\ServerCare\WebGrab\guide\guide.xml not found   ... created a new one ... 

no epg update requested!


Starting MDB Postprocess .. Version: 1.5

LocalMDB file C:\ProgramData\ServerCare\WebGrab\mdb\mdb.xml not found   ... created a new one ... 
XMLTV file - C:\ProgramData\ServerCare\WebGrab\guide\guide.xml - doesn't exist
Check filename and location!
Cannot run MDB postprocessor

MDB Postprocessor result :

Mdb data found for 0 out of 0 Movies-candidates
Mdb data found for 0 out of 0 Series-candidates
In 00:00:00.0450017 seconds

Finished Postprocess MDB
My config file:

Code: Select all

<!--?xml version="1.0"?-->
<settings>
  <!-- for detailed info about the settings see http://webgrabplus.com/node/30 -->
  <filename>C:\ProgramData\ServerCare\WebGrab\guide\guide.xml</filename>
  <mode></mode>
  <postprocess grab="n" run="y">mdb</postprocess>
  <logging>on</logging>
  <retry time-out="5">4</retry>
  <timespan>2</timespan>
  <update>f</update>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="ARD" xmltv_id="Das Erste">Das Erste</channel>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="ZDF" xmltv_id="ZDF">ZDF</channel>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="RTL" xmltv_id="RTL">RTL</channel>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="PRO7" xmltv_id="ProSieben">ProSieben</channel>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="SAT1" xmltv_id="Sat.1">Sat.1</channel>
    <channel update="i" site="tvtv.de.xmltv_ns" site_id="RTL2" xmltv_id="RTL II">RTL II</channel>
</settings>
Attachments
epg.PNG
epg.PNG (114.61 KiB) Viewed 24033 times

Malyngo
Posts: 70
Joined: Wed Sep 08, 2010 8:25 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Malyngo » Tue Aug 26, 2014 10:47 am

This is my WG++ config:
https://onedrive.live.com/?cid=EDD93823 ... 8E%2126076

Code: Select all

<?xml version="1.0"?>
<!-- Configuration file for WebGrab+Plus, the incremental Electronic-Program-Guide web grabber
by Jan van Straaten, December 2011
Version V1.1.1 -->

<settings>
  <filename>C:\ProgramData\ServerCare\WebGrab\guide.xml</filename>
  <mode>m</mode>
  <postprocess run="n" grab="y">mdb</postprocess>
  <proxy>automatic</proxy>
  <user-agent>Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; yie9)</user-agent>
  <logging>on</logging>
  <retry time-out="5">4</retry>
  <skip>noskip</skip>
  <timespan>9</timespan>
  <update></update>

  <channel update="i" site="tvtv.de.onscreen" site_id="ARD" xmltv_id="Das Erste">Das Erste HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="ZDF" xmltv_id="ZDF">ZDF HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="S3" xmltv_id="SWR Fernsehen">SWR Fernsehen BW</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="SAT1" xmltv_id="Sat.1">Sat.1 HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="RTL" xmltv_id="RTL">RTL HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="KAB" xmltv_id="kabel eins">kabel eins HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="PRO7" xmltv_id="ProSieben">Pro7 HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="PCF" xmltv_id="Syfy Universal">Syfy HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="RTL2" xmltv_id="RTL II">RTL2 HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="VOX" xmltv_id="VOX">VOX HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="FOX" xmltv_id="FOX">FOX HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="TCM" xmltv_id="TNT Film">TNT Film HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="TNT" xmltv_id="TNT Serie">TNT Serie HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="3SAT" xmltv_id="3sat">3sat</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="ARTE" xmltv_id="ARTE">arte</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="SUPER" xmltv_id="Super RTL">Super RTL HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="TL5" xmltv_id="Tele 5">Tele 5</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="DMAX" xmltv_id="DMAX">DMAX HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="FES" xmltv_id="Einsfestival">Einsfestival</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="MUX" xmltv_id="EinsPlus">EinsPlus</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="ZDNEO" xmltv_id="ZDFneo">zdf_neo</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="ZTK" xmltv_id="ZDFkultur">zdf.kultur</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="ZDI" xmltv_id="ZDFinfo">ZDFinfo</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="PR7M" xmltv_id="ProSieben Maxx">Pro7 Maxx HD</channel>
  <channel update="i" site="tvtv.de.onscreen" site_id="RTLN" xmltv_id="RTL NITRO">RTL NITRO HD</channel>
  
</settings>
The tvtv.de ini I am using is this one, that I have adapted to onscreen episode information:
https://onedrive.live.com/?cid=EDD93823 ... 8E%2126069

I am seeing the same as you, no series information in the detail window.
To have that, I guess you would have to add it to the description.
To do that, you have to do something like this in the ini:

Code: Select all

temp_1.modify {calculate(not="" format=D2)|}
temp_2.modify {calculate(not="" format=D2)|}

episode.modify {addend('temp_1' not="")|S'temp_1'}
episode.modify {addend('temp_2' not="")|E'temp_2'}

description.modify {addend|(}
description.modify {addend('temp_1'not="")|S'temp_1'}
description.modify {addend('temp_2'not="")|E'temp_2'}
description.modify {addend|) }
The last four lines are new, the first few lines are just for reference, where you have to put those 4 lines. (This is basically from Christophs tvtoday ini, with a few modifications, so credit should go to him)
Attachments
2half.PNG
2half.PNG (33.67 KiB) Viewed 24024 times

raki
Posts: 24
Joined: Fri Oct 04, 2013 9:33 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by raki » Tue Aug 26, 2014 10:49 am

Thanks! First of all, the grabbing itself works now (I did forget to update WebGrabber+ and after that I somehow messed up the config file). I'm gonna have a look at the description settings, thanks!

Edit: Would be interesting if there is any information about the different settings in the ini file and how Argus uses them...

Edit2: This works almost as I want:

Code: Select all

subtitle.modify {addend|'('}
subtitle.modify {addend('temp_1'not="")|S'temp_1'}
subtitle.modify {addend('temp_2'not="")|E'temp_2'}
subtitle.modify {addend|')' }
Problem is: In the overview the episode information are show twice (see attachment).

The question are:
- How does Argus uses the information in the guide.xml file to show them in the scheduler? For example the description and the actors are just put together. I would like to include empty lines or title lile "Actor: XXXXX" next line "Genre: XXXX".

- Is it possible to tell Argus to just import the given fields and keep the rest from the DVB-EPG? For example most of the time the DVB-EPG description is better, just the season and episode information are missing. So I would like to import that just info.

- Why does Argus not recognize the season and episode information from the DVB-EPG? For example Sky provides Season and Episode. How can I get Argus to recognize that (with minimum effort). The description (and all other information) Sky provides is really good and I'd like to keep those.


Thanks!
Attachments
epg.PNG
epg.PNG (95.34 KiB) Viewed 24019 times

Malyngo
Posts: 70
Joined: Wed Sep 08, 2010 8:25 am

Re: WebGrab+Plus and Argus - Supporter Thread

Post by Malyngo » Tue Aug 26, 2014 7:58 pm

How does Sky provide the Episode Info?
Probably in the description? I know TNT Film and Fox do this, they have this information in the description, in the form of "Staffel 1, Folge 13" or similar. The problem is, that Argus would have to parse the description for that, and this would have to be done for different languages, and different styles in the description (One German channel has it in the form of roman numbers in there, IV/12 means S04E13)

Hm, maybe Argus will keep the description if there is none provided in the XMLTV file?

Post Reply

Who is online

Users browsing this forum: No registered users and 13 guests