FeedWordPress 0.96 is loose

Posted on by

Update 2007-11-21: FeedWordPress 0.96 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

0.96 is now available for download.[1]

[2]: http://projects.radgeek.com/download/feedwordpress-0.96.tar.gz

Version 0.96 is a feature-adding release. If there were any bugs that I eliminated between version 0.95 and now, I wasn’t diligent enough to write them down, and I don’t remember any. But I have added three features. Two are minor additions for consistency’s sake or for a minor convenience; one is a bit more significant.

* **FEATURE:** for completeness’s sake, there is now a feed setting,
`hardcode url`, that allows you to set the URI for the front page
of a contributor’s website manually (that is, prevent it from being
automatically updated from the feed channel link on each update). To
set the URI manually, put a line like this in the Link Notes section
of a feed:

hardcode url: yes

You can also instruct FeedWordPress to use hardcoded URIs by default
on all feeds using Options –> Syndication

* **FEATURE:** by default, when FeedWordPress finds new syndicated posts,
it (1) publishes them immediately, (2) turns comments off, and (3)
turns trackback / pingback pings off. You can now alter all three
default behaviors (e.g., to allow pings on syndicated posts, or to
send newly-syndicated posts to the draft pile for moderation) using
Options –> Syndication

* **FEATURE:** support has been added for enclosures in RSS 2.0 and Atom
0.6+ newsfeeds. WordPress already supports adding enclosures to an
item; FeedWordPress merely gets the information on the enclosure
from the feed it is syndicating and plugs that information directly
into the WordPress database so that (among other things) that post
will have its enclosure listed in your blog’s RSS 2 newsfeed.

Note that enclosure support requires using the optional MagpieRSS
upgrade (i.e., replacing your `wp-includes/rss-functions.php` with `OPTIONAL/wp-includes/rss-functions.php` from the FWP archive)

If you’re interested (for example, because you’re writing PHP filters and want
to know how all this is represented), here’s how it’s currently done:

Enclosures are stored in the array $item[3] as suggested
at . So the URL of the first
enclosure is $item[4][5][6]; the length is
$item[7][8][9]; and the type is
$item[10][11][12]

Note that these are hacked-in solutions for inherited problems with
MagpieRSS as of version 0.7. They are not guaranteed to be
forward-compatible when/if Kellan solves these problems in the official
Magpie branch in the future. If you have filters (for example) that
depend on either categories or enclosures working as they currently do,
keep an eye on the ChangeLog in future releases.

and enjoy.[13] Feel free to me know what you think(http://www.radgeek.com/contact)!

FeedWordPress 0.95 is loosed upon the world

Posted on by

Update 2007-11-21: FeedWordPress 0.95 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

0.95 is now available for download.[1]

[2]: http://projects.radgeek.com/download/feedwordpress-0.95.tar.gz

0.95 squashes a bug that I discovered, to my irritation, in the field at Blogs[3], and adds some significant features that may be of interest. Author aliases are now supported, and you can now decide whether or not FeedWordPress will create new authors or new categories when it encounters author names or category names that are not yet in the WordPress database. These features can even be used to provide some simple filtering functionality without having to write a PHP filter. For more information, see the change log and the documentation(http://projects.radgeek.com/feedwordpress).

Blogs: http://www.feministblogs.org/

We are slowly moving towards 1.0 (which, incidentally, I don’t intend to call 1.0 — because, roughly, once it reaches that point it will be mature software and will be unlikely to make any milestone changes of the sort that would make a major version number make sense; the version will most likely be named after the date that it was released).

Here’s a rough outline of what there is left for me to do between now and the 1.0 release:

1. Bug fixes (investigating bugs that users are reporting to me now, and squashing any new ones that come up)

2. Adding some more global options (e.g., allowing users to globally set the default post status, comment status, and ping status for syndicated posts)

3. Support for RSS enclosures.

4. Adding a proper interface for editing FeedWordPress-specific settings for Contributor links; this special editing interface would be reached through the “Edit” link under Links –> Syndicated.

Is there anything important that I’m missing? Let me know what you think.

#### Change Log: from 0.91 to 0.95 ####

* BUG FIX: Fixed an obscure bug in the handling of categories: categories
with trailing whitespace could cause categories with duplicate names to
be created. This no longer happens. While I was at it I tightened up the
operation of FeedWordPress::lookup_categories() a bit in general.

* FEATURE DEPRECATED: the feed setting `hardcode categories` is now
deprecated in favor of `unknown categories` (see below), which allows
you to strip off any syndication categories not already in your database
using `unknown categories: default` or `unknown categories: filter`. If
you have `hardcode categories: yes` set on a feed, this will be treated
as `unknown categories: default` (i.e., no new categories will be added,
but if a post doesn’t match any of the categories it will be added in
the default category–usually “Uncategorized” or “General”).

* FEATURE: You can now set global defaults as to whether or not
FeedWordPress will update the Link Name and Link Description settings
for feeds automatically from the feed title and feed tagline. (By
default, it does, as it has in past versions.) Whether this behavior is
turned on or off, you can still override the default behavior using
feed settings of `hardcode name: yes`, `hardcode name: no`,
`hardcode description: yes`, or `hardcode description: no`.

* FEATURE: Users can now provide one or several “aliases” for an author,
just as they can for a category. For example, to make FeedWordPress
treat posts by “Joseph Cardinal Ratzinger” and “Pope Benedict XVI” as
by the same author, edit the user profile for Pope Benedict XVI and add
a line like this to the “User profile” field:

a.k.a.: Joseph Cardinal Ratzinger

You can add several aliases, each on a line by itself. You can also add
any other text you like to the Profile without interfering with the
aliases.

* FEATURE: Users can now choose how to handle syndicated posts that are
in unfamiliar categories or by unfamiliar authors (i.e., categories or
authors whose names are not yet in the WordPress database). By default,
FeedWordPress will (as before) create a new category (or new author) and
use it for the current post and any future posts. This behavior can be
changed, either for all feeds or for one or another particular feed.

There are now three different options for an unfamiliar author: (1)
FeedWordPress can create a new author account and attribute the
syndicated post to the new account; (2) FeedWordPress can attribute the
post to an author if the author’s name is familiar, and to a default
author (currently, this means the Site Administrator account) if it is
not; (3) FeedWordPress can drop posts by unfamiliar authors and
syndicate only posts by authors who are already in the database.

There are, similarly, two different options for an unfamiliar category:
(1) FeedWordPress can create new categories and place the syndicated
post in them; (2) FeedWordPress can drop the unfamiliar categories and
place syndicated posts only in categories that it is already familiar
with. In addition, FeedWordPress 0.95 lets you choose whether posts that
are in *no* familiar categories should be syndicated (and placed in the
default category for the blog) or simply dropped.

You can set the default behavior for both authors and categories using
the settings in Options –> Syndication. You can also set different
behavior for specific feeds by adding the `unfamiliar author` and / or
`unfamiliar categories` settings to the Link Notes section of a feed:

unfamiliar author: (create|default|filter)
unfamiliar categories: (create|default|filter)

A setting of `unfamiliar author: create` will make FeedWordPress create
new authors to match unfamiliar author names *for this feed alone*. A
setting of `unfamiliar author: default` will make it assign posts from
unfamiliar authors to the default user account. A setting of
`unfamiliar author: filter` will cause all posts (from this feed alone)
to be dropped unless they are by an author already listed in the
database. Similiarly, `unfamiliar categories: create` will make
FeedWordPress create new categories to match unfamiliar category names
*for this feed alone*; `unfamiliar categories: default` will cause it
to drop any unfamiliar category names; and
`unfamiliar categories: filter` will cause it to *both* drop any
unfamiliar category names *and* to only syndicate posts that are placed
in one or more familiar categories.

These two new features allow users to do some coarse-grained filtering
without having to write a PHP filter. Specifically, they offer an easy
way for you to filter feeds by category or by author. Suppose, for
example, that you only wanted to syndicate posts that your contributors
place in the “Llamas” category. You could do so by setting up your
installation of WordPress so that the only category in the database is
“Llamas,” and then use Options –> Syndication to set “Unfamiliar
categories” to “don’t create new categories and don’t syndicate posts
unless they match at least one familiar category”. Now, when you update,
only posts in the “Llamas” category will be syndicated by FeedWordPress.

Similarly, if you wanted to filter one particular feed so that only
posts by (for example) the author “Earl J. Llama” were syndicated to
your site, you could do so by creating a user account for Earl J. Llama,
then adding the following line to the settings for the feed in Link
Notes:

unfamiliar author: filter

This will cause any posts from this feed that are not authored by Earl
J. Llama to be discarded, and only the posts by Earl J. Llama will be
syndicated. (If the setting is used on one specific feed, it will not
affect how posts from other feeds are syndicated.)

FeedWordPress 0.91

Posted on by

Update 2007-11-21: FeedWordPress 0.91 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

0.91[1] is now available for download.

0.91 is a release that squashes a few bugs and adds a few useful features. Really, it adds enough useful features that it’s worth an increment of 0.1 rather than 0.01, but it’s not where I want FeedWordPress to be for version 1.0 yet, and it certainly hasn’t regressed to 0.10, so we’ll just call it 0.91 and grit our teeth at the vagaries of the decimal system for the time being.

[2]: http://projects.radgeek.com/download/feedwordpress-0.91.tar.gz

it a whirl[3], and don’t hesitate to me(http://www.radgeek.com/contact) with any questions, comments, applause, brickbats, etc. I’ll be updating the documentation shortly to reflect the changes since 0.9. In the meantime, here’s the highlight reel:

### Features Added ###

1. **Category aliases:** many times, when you have an aggregation website with many contributors, you will end up with several post categories that mean more or less the same thing but have slightly names (for example, Blogs(http://www.feministblogs.org/) ended up with “Feminism,” “Feminism, sexism, etc.”, “feministy stuff,” “Gender Issues”, “Women’s Issues”, and so on). You can now designate “aliases” for a category so that posts with their category set to *either* the actual name of the category *or* to any of its aliases will be placed in that category. To add an “alias” for a category, (1) log in to the WordPress Dashboard and (2) go to Manage –> Categories, (3) click the “Edit” link for the category that you want to add the alias for, and (4) add a line like the following, on a line by itself, to the “Description” field:

a.k.a.: Other category name

(Of course, you should substitute the actual alias for “Other category name”.)

2. **Hard-coded titles and taglines:** By default, FeedWordPress automatically updates the Link Name and Link Description fields on a syndicated feed using the title and the tagline or description contained in the newsfeed. (So, for example, if a contributor changes the title of her blog, the new title will automatically be reflected in your “Contributors” section the next time that FeedWordPress updates its content.) If you want to *stop* FeedWordPress from automatically updating these fields for a particular feed (for example, because you want to use an abbreviated title or tagline for reasons of space), you can now do so by (1) going to Links –> Syndicated, (2) clicking the “Edit” link for the feed that you want to change the title or tagline for, (3) entering your hard-coded title and/or tagline in the Link Name and/or Link Description field, then (4) adding a line like this to the Link Notes section:

hardcode name: yes

Or a line like this:

hardcode description: yes

`hardcode name` will ensure that FeedWordPress uses your hand-entered Link Name; `hardcode description` will ensure that FeedWordPress uses your hand-entered Link Description; adding both lines will ensure that FeedWordPress uses both your hand-entered Link Name *and* your hand-entered Link Description.

3. **Ignoring syndicated categories:** By default, FeedWordPress places syndicated posts in the same categories that the newsfeed they were syndicated from places them in (so, for example, if one of your contributors placed a post in the “News” category on her blog, FeedWordPress would place the syndicated post in the “News” category on your blog; if the “News” category did not yet exist, FeedWordPress would create it automatically). If you want FeedWordPress to place posts from a particular feed in *only* the categories you manually set, and ignore the categories on the syndication feed, you can now do so by (1) going to Links –> Syndicated, (2) clicking the “Edit” link for the feed that you want to change, then (3) adding a line like this to the Link Notes section:

hardcode categories: yes

### Bug fixes ###

1. If you got an error message like this when you tried to add a new feed for syndication:

Fatal error: Call to undefined function: curl_init() in /…/feedwordpress.php on line 1353

… you can fix the problem by upgrading to 0.91[4]. FeedWordPress no longer depends on the PHP extension(http://us2.php.net/manual/en/ref.curl.php). It now uses the Snoopy module, which comes packaged with all WordPress 1.5 installations, to fetch data from any URI that it needs to fetch data from.

2. Manual editing of feed settings using the Link Notes section could introduce a couple of bugs under some web browsers–for example, by tacking a strange character (a carriage return, or `”\r”`, specifically). This has been fixed: carriage returns (and, indeed, any trailing whitespace) will no longer screw up your settings.

Thanks to Ray Lischner for pointing this bug out.

3. If you used FeedWordPress to syndicate posts on a blog that you also posted to directly through the WordPress Dashboard, you may have noticed that FeedWordPress 0.9 disabled WordPress’s automatic formatting filter. (So if FeedWordPress was activated, any posts that you made directly into WordPress wouldn’t have `

…

` tags wrapped around paragraphs unless you hand-coded them in.) This was not so much a bug as a regrettable side-effect of the way I initially handled the problem of keeping WordPress’s filters from mangling the HTML on syndicated posts. In any case, it is now fixed: FeedWordPress stops the normal formatting filters from working *only* on syndicated posts (which are already in HTML), and *not* on any posts or pages that you posted directly to your site.

Thanks to Alun for reminding me to fix this.

4. If a newsfeed provides an excerpt or summary for a particular post, 0.91[5] now uses it as the excerpt for the post on your aggregator site. (In FeedWordPress 0.8 and 0.9, the excerpt was discarded.) If no excerpt is provided, FeedWordPress will generate one from the post content.

### Issues not yet fixed ###

This is a fallen world, and unfortunately there are a few bugs on the docket that have not yet been squashed:

1. **Blank page with no options under Options –> Syndication:** This is a bug with WordPress 1.5, not with FeedWordPress. It results from a bug in WordPress 1.5’s Dashboard that makes it possible for the custom Options pages that plugins define to collide with one another. You may be able to solve the problem by deactivating all plugins except for FeedWordPress. (You can then re-activate plugins, one at a time, to see which one causes the collision.) Alternatively, you an updated version of `wp-admin/menu-header.php`(http://trac.wordpress.org/file/trunk/wp-admin/menu-header.php) and upload it to the `wp-admin` directory of your WordPress installation. This bug will by fixed in WordPress 1.5.1.

2. **Some feeds with alternative character sets get mangled:** FeedWordPress mostly handles alternative character sets fairly well, but its ability to do so depends on support from (1) MagpieRSS, (2) PHP, and (3) in PHP 4.x, the iconv library. This can make the whole affair pretty complicated, and at least one user has reported trouble (with incorrect conversion of Cyrillic characters encoded in windows-1251). This has not yet been fixed, because the multiple layers of software involved make it a non-trivial problem to solve. In the meantime, if you have been getting incorrect conversions of the character set on one of your feeds, you can work around the problem by restoring the version of `wp-includes/rss-functions.php` which comes with WordPress 1.5, instead of using the optional upgrade included with FeedWordPress.

FeedWordPress 0.9

Posted on by

Update 2007-11-21: FeedWordPress 0.9 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

0.9 is now available for download.[1]

[2]: http://projects.radgeek.com/download/feedwordpress-0.9.tar.gz

The current update does a couple of things.

First, it definitively slays the text config file: there are *no* options at all involved in normal operation that require editing either a configuration file or the PHP source to change. This was a guiding principle in 0.8, but I left a couple of the more technical items (such as whether or not to log updates into the PHP log) as define() statements in the PHP source. No longer: anything that *can* be set, can be set through the WordPress Dashboard at Options –> Syndication, or on individual feeds in Links –> Syndicated.

Second, it provides hooks through the WordPress plugin architecture(http://projects.radgeek.com/feedwordpress/use#plugin-api) that you can use to supplement FeedWordPress’s behavior on updates, and–somewhat more likely to be immediately useful to you–*filter* incoming items, and (if you so desire) either modify their contents or filter the post out entirely. of these hooks(http://projects.radgeek.com/feedwordpress/use#plugin-api) is a bit sketchy as of 2005-03-29, but I hope to flesh it out in the near future. For now, here’s a quick example that you might be able to turn to your own purposes: say that you want to filter out any posts entitled “Friday Cat Blogging”. You could define a filter like this:

[3] ) ) ) !== false) :
$item = NULL;
endif;
endif;
return $item;
}
?>

Once this filter is Activated from the WordPress Dashboard, any incoming posts with ‘Friday Cat Blogging’ in the title will be filtered out of the feed.

I’m planning on adding some filtering features to upcoming updates of FeedWordPress that don’t require any PHP coding, but they will be pretty simple and inflexible in nature–I’m not about to invent any convoluted new filtering syntax to do complicated things without PHP scripting. If your filtering needs are complicated enough that you need a syntax with a lot of expressive power, then PHP’s syntax is a pretty good bet. If they’re less complicated, then there’s no reason why the filtering features accessible to you need to be able to do things like match arbitrary fields against regular expressions. But I think there are good, simple ways to provide some basic features such as category-filtering and author-filtering from within the Dashboard.

I think this leaves just a few items on the to-do list prior to releasing a package I’d be willing to call “1.0”:

1. Simple filtering by author or by category without any scripting

2. An interface for editing the properties of syndicated Links, rather than simply dumping the user off at the normal “Edit Link” page. (This should, for example, provide a nice way to edit feed settings without having to dig around in the big textarea for the Link Notes.

3. … is there anything else that is definitely needed? These are the main items I want to tackle, but if there’s something that would be helpful to you, let me know…

Initial Public Offering

Posted on by

Update 2007-11-21: FeedWordPress 0.8 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

The initial public release of FeedWordPress is now for download[1]. Enjoy, and feel free to me[2] in private or in the comments with any questions, comments, applause, or brickbats.

[3]: http://projects.radgeek.com/download/feedwordpress-0.8.tar.gz
[4]: http://www.radgeek.com/contact

FeedWordPress is an Atom/RSS aggregator plugin for [5][6] 1.5. You give it a list of websites with Atom or RSS newsfeeds and it will syndicate posts from them into your WordPress weblog. It uses [7][8] to parse the feeds and some simple transformations to store the data as a post in the WordPress database. It’s a classic case of “itch-scratch ware”–I wrote it specifically because I needed something more flexible than [9][10] to run Blogs[11]. But Blogs[12] isn’t the only aggregator site in the world (let alone the only *potential* one), and I’m making the code public in the hopes that y’all might find it useful and pleasant to work with too.

[13]: http://www.wordpress.org/
[14]: http://magpierss.sourceforge.net/
[15]: http://www.planetplanet.org/
Blogs: http://www.feministblogs.org/

One of the things you’ve got to do in life is keep track of the people that you owe. FeedWordPress is free software, which makes use of code derived from more than one other free software project. The core aggregation code is adapted from an experimental PHP script (wp-rss-aggregate.php) by Elliot McCrae(http://www.laughingmeme.org/); auxiliary modules are adapted from a URL class by Minoukadeh(http://www.keyvan.net/), and the Feed Finder(http://diveintomark.org/projects/feed_finder/) by Pilgrim(http://diveintomark.org/). FeedWordPress might have been possible without their help, but it sure would have been more of a pain in the ass. Thanks, everyone, and here’s to free software!