%META:TOPICINFO{author="TWikiContributor" date="1531550015" format="1.1" version="11"}% ---+!! Headlines Plugin
%TOC{ title="Page contents" depth="2" }%
%SHORTDESCRIPTION% ---++ Description This plugin displays RSS and ATOM feeds from news sites. Use it to build news portals that show headline news. __Notes:__ * [[Wikipedia:RSS][RSS]] (Rich Site Summary) is an XML standard used to publish frequently updated works such as blog entries, news headlines and audio. * [[Wikipedia:Atom_%28standard%29][Atom]] is an XML syndication format used for web feeds. * Syndic8.com at http://www.syndic8.com/ lists many RSS and ATOM feeds. #SyntaxRules ---++ Syntax Rules =%HEADLINES{"..."}%= | *Parameter* | *Explanation* | *Default* | | ="..."= | Source of RSS or ATOM feed; this can be an url (starting with http) or a web.topic location for internal feeds | None; is required | | =href="..."= | (Alternative to above) | N/A | | =refresh="60"= | Refresh rate in minutes for caching feed; ="0"= for no caching | Global =REFRESH= setting | | =limit="12"= | Maximum number of items shown | Global =LIMIT= setting | | =header="..."= | Header. May include these variables: %BR% \ - =$channeltitle=, =$title=: title of channel (channel.title) %BR% \ - =$channellink=, =$link=: link of channel (channel.link) %BR% \ - =$channeldescription=, =$description=: description (channel.description) %BR% \ - =$channeldate=, =$date=: publication date of the channel (channel.pubDate) %BR% \ - =$rights=: copyrights of the channel (channel.copyright) %BR% \ - =$imagetitle=: title text for site (image.title) %BR% \ - =$imagelink=: link for site (image.link) %BR% \ - =$imageurl=: URL of image (image.url) %BR% \ - =$imagedescription=: description of image (image.description) \ | Global =HEADER= setting | | =format="..."= | Format of one item. May include these variables: %BR% \ - =$title=: news item title (item.title) %BR% \ - =$link=: news item link (item.link) %BR% \ - =$description=: news item description (item.description) %BR% \ - =$date=: the publication date (item.pubDate, item.date) %BR% \ - =$category=: the article category (item.category) \ | Global =FORMAT= setting | | =newline="$br"= | Convert newlines in feed; ="$br"= becomes =<br />= tag | ="$n"= (newline) | | =filter="..."= | Filter out content from feed; for example to delete an encoded break tag specify ="&lt;br&gt;"= | =""= | | =touch="..."= | Touch (edit/save) topics if the feed has updates. Specify a comma-space delimited list of !TopicNames or !Web.TopicNames, such as ="%TOPIC%, !NewsLetter"=. Useful to send out newsletter using MailerContrib, showing new feeds since last newsletter. To update feeds, visit topics with feeds in regular intervals (using cron with wget or the like). | N/A | The =header= and =format= parameters might also use variables rendering the =dc=, =image= and =content= namespace information. Note, that only bits of interest have been implemented so far and those namespaces might not be implemented fully yet. ---+++ Rendering the =dc= namespace The following variables are extracting the =dc= namespace info, that could be used in =header= and =format=. Note that some of the variables are already used above. This is done by purpose to use different feeds with the same formatting parameters. If there's a conflict the non-dc tags have higher precedence, i.e. a =<title> content </title>= is preferred over =<dc:title> content </dc:title>=. * =$title=: channel/article title (dc:title) * =$creator=: channel creator (dc:creator) * =$subject=: subject text; this will also add an image according to the subject hash list, see [[#Subject_Images][above]] (dc:subject) * =$description=: ... (dc:description) * =$publisher=: the channel/article publisher (dc:publisher) * =$contributor=: ... (dc:contributor) * =$date=: ... (dc:date) * =$type=: ... (dc:type) * =$format=: ... (dc:format) * =$identifier=: ... (dc:identifier) * =$source=: ... (dc:source) * =$language=: ... (dc:language) * =$relation=: ... (dc:relation) * =$coverage=: ... (dc: coverage) * =$rights=: ... (dc: rights) ---+++ Rendering the =image= namespace An =image:item= is converted into an <img> tag using the following mappings: * =src=: image url (rdf:about attribute of the image.item tag) * =alt=: image title (title) * =width=: image width (image:width) * =height=: image height image:height) ---+++ Rendering the =content= namespace The variable =$content= is referring to the =<content:encoding> content </content:encoding>=. ---++ Examples ---+++ Slashdot News Write %HEADLINES{ "http://slashdot.org/slashdot.rdf" header="*[[$link][$title]]:* $description" format="$t* [[$link][$title]]" limit="4" }% to get the latest Slashdot news as a bullet list format: %HEADLINES{ "http://slashdot.org/slashdot.rdf" header="---+!! [[$link][$title]]$n $description" format="$t* [[$link][$title]]" limit="4" }% ---++ Plugin Settings Plugin settings are stored as preferences settings. Do __not__ change the settings here, they are here only for illustration purposes showing the default values. Define the settings in [[%LOCALSITEPREFS%]]. For example, to customize the =HEADLINESPLUGIN_USERAGENTNAME= setting, add a =* Set HEADLINESPLUGIN_USERAGENTNAME = ...= bullet in [[%LOCALSITEPREFS%]]. * Refresh rate in minutes for cached feeds. Set to =0= to disable caching: * Set HEADLINESPLUGIN_REFRESH = 60 * Maximum number of items shown: * Set HEADLINESPLUGIN_LIMIT = 100 * Use LWP::UserAgent if set to =1=, or fallback to TWiki's internal =getUrl()= method if set to =0=: * Set HEADLINESPLUGIN_USELWPUSERAGENT = 1 * Timeout fetching a feed using the LWP::UserAgent: * Set HEADLINESPLUGIN_USERAGENTTIMEOUT = 20 * Name of user agent: * Set HEADLINESPLUGIN_USERAGENTNAME = TWikiHeadlinesPlugin/2011-07-08 * Default header: (variables are explained in the syntax rules) * Set HEADLINESPLUGIN_HEADER =
$n---+!! $title
$date
$description
$rights
 * Default format of one item: (variables are explained in the syntax rules) * Set HEADLINESPLUGIN_FORMAT =
$title
$n$date $creator $subject $n
$description
 * Allow HTML in header and format parameters of the HEADLINES variable. If set to =0=, HTML is *not* allowed those in parameters. This is to guard against Cross-Site Scripting (XSS) attacks. The HEADLINESPLUGIN_HEADER and HEADLINESPLUGIN_FORMAT plugin settings allow HTML regardless of this setting. * Set HEADLINESPLUGIN_ALLOWHTML = 0 * Values taken from configure: (only supported if CPAN:LWP is installed) * =$TWiki::cfg{PROXY}{HOST}= - proxy host, such as ="proxy.example.com";= * =$TWiki::cfg{PROXY}{PORT}= - proxy port, such as ="8080";= * =$TWiki::cfg{PROXY}{SkipProxyForDomains}= - domains excluded from proxy, such as ="intra.example.com, bugs.example.com";= ---++ Style Sheets The default HEADER and FORMAT settings use the following styles. See the [[%ATTACHURL%/style.css][style.css]] file defining the default CSS properties (indentation illustrates enclosure). * =headlinesRss=: output of the HeadlinesPlugin (div) * =headlinesChannel=: channel header (div) * =headlinesLogo=: channel logo (div) * =headlinesTitle=: channel title (div) * =headlinesDate=: channel date (div) * =headlinesDescription=: channel description (div) * =headlinesRight=: channel copyright (div) * =headlinesArticle=: one news item (div) * =headlinesTitle=: article title (div) * =headlinesDate=: article date (span) * =headlinesCreator=: author of article (span) * =headlinesSubject=: subect category of the article (span) * =headlinesText=: article text (div) ---++ Plugin Installation Instructions This is a pre-installed plugin. TWiki administrators can upgrade the plugin as needed on the TWiki server. %TWISTY{ mode="div" showlink="Show details %ICONURL{toggleopen}% " hidelink="Hide details %ICONURL{toggleclose}% " }% __Note:__ You do not need to install anything on the browser to use this plugin. The following instructions are for the administrator who installs the plugin on the TWiki server. * For an __automated installation__, run the [[%SCRIPTURL{configure}%][configure]] script and follow "Find More Extensions" in the in the __Extensions__ section. * Or, follow these __manual installation__ steps: * Download the ZIP file from the Plugins home (see below). * Unzip ==%TOPIC%.zip== in your twiki installation directory. Content: | *File:* | *Description:* | | ==data/TWiki/%TOPIC%.txt== | Plugin topic | | ==pub/TWiki/%TOPIC%/style.css== | Default CSS | | ==lib/TWiki/Plugins/%TOPIC%.pm== | Plugin Perl module | | ==lib/TWiki/%TOPIC%/Core.pm== | Plugin core | * Set the ownership of the extracted directories and files to the webserver user. * Make sure the dependencies listed in the table below are resolved.
NameVersionDescription
Digest::MD5>=2.33Required. Download from CPAN:Digest::MD5
LWP::UserAgent>=5.803Optional. Download from CPAN:LWP::UserAgent
* Plugin __configuration and testing__: * Run the [[%SCRIPTURL{configure}%][configure]] script, enable the plugin in the __Plugins__ section * Configure the plugin: See plugin settings above. * Test if the installation was successful: See example above. %ENDTWISTY% ---++ Plugin Info * One line description, shown in the TextFormattingRules topic: * Set SHORTDESCRIPTION = Show headline news in TWiki pages based on RSS and ATOM news feeds from external sites %TABLE{ tablewidth="100%" columnwidths="170," }% | Plugin Author: | TWiki:Main.PeterThoeny, TWiki:Main.MichaelDaum | | Copyright: | © 2002-2018 Peter Thoeny, [[http://twiki.org/][TWiki.org]];%BR% © 2002-2018 TWiki:TWiki.TWikiContributor; %BR% © 2005-2007 Michael Daum wikiring.de | | License: | GPL ([[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]]) | | Plugin Version: | 2018-07-13 | %TWISTY{ mode="div" showlink="Show Change History %ICONURL{toggleopen}%" hidelink="Hide Change History %ICONURL{toggleclose}% " }% %TABLE{ tablewidth="100%" columnwidths="170," }% | 2018-07-13: | TWikibug:Item7846: Sanitize parameters; add HEADLINESPLUGIN_ALLOWHTML setting; sanitize malicious scripts in feed; remove non-functional Wired feed example | | 2018-07-05: | TWikibug:Item7841: Copyright update to 2018 | | 2016-01-08: | TWikibug:Item7708: Copyright update to 2016 | | 2015-11-06: | TWikibug:Item7697: New filter and newline parameters; fix Atom feed parser to support Google Alerts | | 2015-07-28: | TWikibug:Item7678: HeadlinesPlugin not working well on Fast CGI and mod_perl | | 2015-01-10: | TWikibug:Item7604: Switch from GPL v2 to v3 | | 2013-11-18: | TWikibug:Item7385: Encode "[" and "]" to avoid problems with TWiki =[[...][...]]= links; code-refactor to use registerTagHandler and OO style. | | 2013-02-16: | TWikibug:Item7091: Use TWISTY in installation instructions section and change history | | 2013-01-29: | TWikibug:Item7131: HeadlinesPlugin doesn't match pattern in multiple lines. | | 2012-12-10: | TWikibug:Item7070: HeadlinesPlugin doesn't support multiple disks. | | 2012-11-13: | TWikibug:Item7016: Restore changes for earlier versions with LWP | | 2012-11-13: | TWikibug:Item7020: Categorize TWiki Variable | | 2012-10-31: | TWikibug:Item7016: Utilize <guid> value when it is available | | 2012-09-05: | TWikibug:Item6837: Doc fixes | | 2011-07-17: | TWikibug:Item6764: Add VarHEADLINES variable documentation; doc improvements; setting NO_PREFS_IN_TOPIC | | 2011-07-08: | TWikibug:Item6725: Change global package variables from "use vars" to "our" | | 2010-05-16: | TWikibug:Item6433: More doc improvements | | 2010-04-25: | TWikibug:Item6433: Doc fix: Changing TWIKIWEB to SYSTEMWEB | | 2010-02-27: | TWikibug:Item6313: Fixed bug in ATOM feed with =<link ...></link>= instead of =<link ... />= -- Peter Thoeny | | 2009-09-30: | fixed bug in lastBuildDate of feeds affecting touch parameter functionality -- Peter Thoeny | | 2009-08-29: | added touch parameter -- Peter Thoeny | | 12 Feb 2009: | {PROXY}{HOST} supports domain with and without protocol -- Peter Thoeny | | 06 Feb 2009: | added {PROXY}{SkipProxyForDomains} configure setting, added USERAGENTNAME plugin setting -- Peter Thoeny | | 11 Dec 2008: | added {PROXY}{HOST} and {PROXY}{PORT} configure settings -- Peter Thoeny | | 13 Sep 2007: | fixed parsing of content:encoded | | 23 Jul 2006: | improved atom parser; if a posting has no title default to 'Untitled' | | 26 Apr 2006: | added lazy compilation | | 10 Feb 2006: | packaged using the TWiki:Plugins/BuildContrib; minor fixes | | 03 Feb 2006: | off-by-one: limit="n" returned n+1 articles; make FORMAT and HEADER format strings more robust | | 23 Jan 2006: | released v2.00 | | 05 Dec 2005: | internal feed urls must be absolute | | 02 Dec 2005: | added web.topic shorthand for internal feeds | | 29 Nov 2005: | fixed CDATA handling | | 21 Nov 2005: | added ATOM support; extended RSS support; added dublin core support; added content support; optionally using LWP to fetch feeds to follow redirections; corrected CPAN dependencies ; recoding special chars from html integer to entity encoding to increase browser compatibility; added css support; use =getWorkArea()= if available | | 11 May 2005: | TWiki:Main.WillNorris: added DevelopBranch compatability | | 31 Oct 2004: | Fixed taint issue by TWiki:Main.AdrianWeiler; small performance improvement | | 29 Oct 2004: | Fixed issue of external caching if mod_perl or SpeedyCGI is used | | 02 Aug 2002: | Implemented caching of feeds, thanks to TWiki:Main/RobDuarte | | 11 Jun 2002: | Initial version (V1.000) | %ENDTWISTY% %TABLE{ tablewidth="100%" columnwidths="170," }% | Perl Version: | 5.8 | | [[TWiki:Plugins/Benchmark][Plugin Benchmark]]: | %SYSTEMWEB%.GoodStyle 100%, %SYSTEMWEB%.FormattedSearch 99.5%, %TOPIC% 94% | | Plugin Home: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC% | | Feedback: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC%Dev | | Appraisal: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC%Appraisal | __Related Topics:__ %SYSTEMWEB%.VarHEADLINES, %SYSTEMWEB%.TWikiPlugins, %SYSTEMWEB%.AdminDocumentationCategory, %SYSTEMWEB%.TWikiPreferences