FeedWordPress 2011.0512: better performance, better troubleshooting, and no more “There may be a bug” white-screens-of-death

2011.0512 is now available for download.(http://downloads.wordpress.org/plugin/feedwordpress.2011.0512.zip) Hooray!

This announcement will actually cover details for a couple of maintenance releases that I’ve pushed out since [1](http://feedwordpress.radgeek.com/blogs/radgeek/2010/09/05/feedwordpress-20100905-bug-fixed-categories-and-tags-now-correctly-assigned). This is mainly a maintenance release, with a number of small tweaks around the edges, most of them for better internal performance or organization, but — if you’re upgrading from 2010.0905 — it includes one important, major performance improvement. And it includes a number of changes which should make it much quicker and much less painful to troubleshoot if you encounter any problems with your feeds. Besides having brought the ChangeLog up to date with the most recent version of the code, here are some of the main things that have changed:

### Changes since Version 2010.0905 ###

exceptionally long GUIDs, FeedWordPress could occasionally get into
a situation where posts with the long GUIDs would be duplicated over
and over again with each update (because FWP failed to store the full
GUID, due to length constraints in the relevant database tables).
Without the full GUID, FWP would not know that the post had already
been syndicated once. This bug has been fixed, and should no longer
produce duplicate posts.

* **HTTP TIMEOUT SETTING:** If you are frequently running into timeout
problems with one or more of the feeds you syndicate, FWP now allows you
to adjust the timeout for HTTP requests using a global or feed-by-feed

* **HTTP GET PARAMETERS:** You can now temporarily or permanently add HTTP
GET parameters to a subscription using an interface in Syndication –>
Feeds & Updates. This is especially helpful for making quick, short-term
changes to a subscription (for example, to pull in all the previous
items from a web service, before settling down to pulling in only newly
updated items). You can also use it to include private parameters with values that you do not wish to make publicly visible when a syndicated post displays the URL of its source feed — such as API keys or passwords.

* **DIAGNOSTICS SYSTEM:** Added several new diagnostics which are useful in
troubleshooting, and established a framework for add-on modules to hook
in with their own diagnostic messages.

* **UI:** Adjusted some internal coding, which should allow for settings
pages and add-ons to properly display multiple category pickers on a
single settings page.

* **PHP4 COMPATIBILITY TWEAKS:** This version makes some tweaks to the handling
of object references which should improve compatibility with older
versions of PHP. (Although, I should note, web hosts that still force
you to run under PHP 4 — in 2011! — are *bad web hosts*.)

* **IMPROVED PERFORMANCE:** This version eliminates a *major* performance drag
that shows up on sites with large numbers of users (due to some poor
decisions about where to place a user query, which caused the user table
to be scanned frequently when it did not need to be). If you experienced
serious problems with CPU load or slow database performance under
2010.0905, which kicked in immediately when FWP was loaded and tended
to disappear immediately if FWP was de-activated, it is likely that
upgrading away from 2010.0905 to the most recent version will resolve
your problem.

### Changes since Version 2011.0211.2 ###

to the Syndication –> Diagnostics section, which should aid in
troubleshooting difficulties with items failing to be imported, posts
failing to be properly inserted into the database, or updates failing to
be recorded. If you have been encountering critical error / bug notices
with a white screen and the message “THERE MAY BE A BUG IN
FEEDWORDPRESS,” followed by an extraordinarily long dump of mostly
incomprehensible diagnostic information, you’ll be happy to know that
the condition causing these notices has been eliminated. In the few
cases where errors may still crop up with database insertions,
FeedWordPress will now produce a significantly more manageable and more
useful diagnostic message.

* **BUGFIX: NEW POSTS FAILING TO APPEAR IN A CLEANLY-INSTALLED FEEDWORDPRESS SYSTEM.** If you encountered a recurring problem with FeedWordPress
failing to import new posts, after a clean install of FeedWordPress
(i.e., not an upgrade from a previous version), this problem may have
been the result of a bug with author-handling which has now been fixed
in the 2011.0512 release. (If the problem does *not* go away with the
upgrade, this version also includes significant improvements to the
Diagnostics system, which will help track down what *is* causing it
in your particular case.)

* **PERFORMANCE:** New handling of update hashes allows FeedWordPress to avoid
a certain kind of infinite loop, caused when two more more different
syndicated feeds each carried a version of the same item (for example,
because it appeared on two different aggregator feeds that you’re
syndicating). In previous versions, when this kind of loop cropped up,
syndicated posts could pile up an indefinitely large number of revisions
— each revision alternating between the version from each of the two
feeds where it appeared — which would, over time, dramatically inflate
the size of the database, and kill the performance of queries on the
post table. This issue has been resolved: revisions of the post that
have been syndicated once will not be re-syndicated over and over again.

* **AUTHOR LISTS:** Lists of authors presented on the Author settings pages
should now be easier to scan through, with author names arranged in
alphabetical order.

* **FEED ITEM DATE PARSING:** More tweaks to make date-time handling more
resilient when feeds provide broken or weird values for the timestamps
on syndicated items. FWP will now attempt to work around unparseable
timezone values.

* **AUTHOR MATCHING:** Now attempts to match author names against the WP login
name in addition to display_name; when creating user record, also fills
in some best-guess values for nickname, firstname and lastname. Also
properly picks up Atom 1.0 author/uri data from feed.

* **COMPATIBILITY:** FeedWordPress has been successfully tested for
compatibility with recent releases of WordPress, up to version 3.1.2.

### As always… ###

As always, if you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or me a line(http://feedwordpress.radgeek.com/contact) to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you’re using, what version of WordPress you’re using it with, which web browser you are using to view the FeedWordPress user interface, and try to tell me, as clearly as possible, (1) what you were trying to do, (2) what the circumstances were, (3) what you expected to see, and (4) what you ended up seeing instead.

Please remember that your generous gifts to project tip jar(http://feedwordpress.radgeek.com/donate) make ongoing development and support for FeedWordPress possible.

Now get on out there, and enjoy!(http://downloads.wordpress.org/plugin/feedwordpress.2011.0512.zip)

FeedWordPress 2010.0905: bug fixed; Categories and Tags now correctly assigned

**Note.** *As with the previous release, before you download or install, please note that this version of FeedWordPress **requires WordPress 3** or higher to operate correctly. Do not install the upgrade unless or until you have made the upgrade to WP3.*

2010.0905 is now available for download.(http://downloads.wordpress.org/plugin/feedwordpress.2010.0905.zip)

I’ve pushed out this release quickly because it contains fixes to an important issue that y’all helped me discover in early user reports back about 2010.0903(http://feedwordpress.radgeek.com/blogs/radgeek/2010/09/03/feedwordpress-20100903-bug-fixes-interface-improvements-lower-memory-load-b). As I mentioned in the release notes, the new release of FeedWordPress has some extensive internal changes to allow for a great deal more flexibility in how it handles Categories and Tags (up to and including support for custom taxonomies provided by plugins). Unfortunately, the changes introduced a bug, due to the fact that the new method used to insert Categories and Tags in WordPress 3 screens the terms to be inserted by the current logged-in user’s capabilities. Which is fine for normal operations within the WordPress administrative interface; but since automatic updates and updates from FeedWordPress cron jobs are typically carried out *without* any user logged in whose credentials could be checked, this meant that several users noticed posts being inserted with only the category Uncategorized, and with no Tags. Which presents a problem.

Fortunately, the problem has been fixed in today’s quick upgrade:

Version 2010.0903 switched over to a new way of assigning categories and
tags as part of its support for handling custom taxonomies.
Unfortunately, the method that it uses is subjected to some checks of
the current user’s capabilities, which creates problems for posts that
are being inserted into the WordPress database when there *is* no
current user logged in (as, for example, when an update is being carried
out from a cron job or automatic update). The result was that posts
from cron jobs and automatic updates ended up with no Categories and no
Tags being assigned. This bug has now been fixed: in 2010.0905, Tags and
Categories should be correctly assigned to all posts, regardless of
whether they were added from manual updates, cron jobs, or automatic

All you need to do to apply the fix is download the newest release and drop it into place in your WordPress plugins directory. As always, if you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or me a line(http://feedwordpress.radgeek.com/contact) to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you’re using, what version of WordPress you’re using it with, which web browser you are using to view the FeedWordPress user interface, and try to tell me, as clearly as possible, (1) what you were trying to do, (2) what the circumstances were, (3) what you expected to see, and (4) what you ended up seeing instead.

and enjoy!(http://downloads.wordpress.org/plugin/feedwordpress.2010.0905.zip).

FeedWordPress 2010.0903: bug fixes, interface improvements, lower memory load & big new features for convenience & uber-geekery

I am happy to announce that 2010.0903 is now available for download.(http://downloads.wordpress.org/plugin/feedwordpress.2010.0903.zip).

**Now stop just a minute.** Before you proceed, be advised: FeedWordPress 2010.0903 has been thoroughly tested for compatibility with WordPress 3.0. It also **requires** WordPress 3.0. If you do not have WordPress 3.0 installed, and aren’t yet ready to make the upgrade, you *should not* attempt to upgrade to this version of FeedWordPress, until you *are* ready to make the upgrade to WP3.

This release includes a big passle of improvements, fixes, and new features. Some of them have been a long time coming; others are new and fancy. For the fixes, you may be most interested in a fix which apparently solves a really vexing vanishing act that Syndicated Sources performed for some users on various versions of Internet Explorer — where a big chunk of the user interface would suddenly disappear as soon as you added your first feed. You may also notice some of the new features, like **subscribing to multiple feeds** or the reductions in memory consumption. Many of the best changes, though you may not even notice — a great deal of the improvements are in low-level interface work, and if I’ve done my job right it will make your life a few minutes more pleasant every day, without you hardly even noticing it. Many of them are improvements under the hood, which are mainly intended for uber-geeks, hotrodders, or — and here is where it matters for the rest of you — and Filter Developers(http://feedwordpress.radgeek.com/wiki/add-ons-and-filters), who will have that much more of a chance to enrich the FeedWordPress ecosystem. But there’s also plenty that should be right out on the surface, for anyone to see, and I hope you enjoy it.

Here’s a breakdown of some of the major changes since the previous release:

* **WORDPRESS 3 REQUIRED:** Please note that this release of FeedWordPress
*requires* WordPress 3.0 or later. If you are currently using a 2.x
branch of WordPress, you will need to upgrade to WordPress 3 before you
can successfully upgrade FeedWordPress.

EXPLORER UI GLITCH APPARENTLY FIXED:** Several users independently
reported a problem with FWP 2010.0623 and various versions of IE. A
problem with the HTML markup caused IE (but not Firefox or
Chrome) to completely hide the Syndicated Sources administration panel
(the main list of currently-syndicated sources, and the main location
for adding new sources, under the Syndication menu item) when a user
added their first syndicated feed. Maddeningly, the glitch seemed to
affect some IE users and not others: I was never able to reproduce the
problem for myself on my own machines. However, the markup of Syndicated
Sources has undergone significant changes and corrections since
2010.0623, and two independent sources who had been having this problem
confirm that they no longer encounter it with the updated version. For
the time being, I am going to declare this bug squashed.

SimplePie:** FeedWordPress now takes some precautions that should help to
better avoid conflicts for users who have installed pluggable versions
of SimplePie for another plugin or theme. (You may not know that you have
done this; but if you’ve been encountering fatal errors indicating that
you cannot redeclare class SimplePie, or something along those lines,
there is now a better chance that those fatal errors will be eliminated.

FeedWordPress is still a memory-hungry little module, especially when
you are dealing with very large feeds. However, users should notice a
significant reduction in memory overloads, especially if they update a
large number of feeds at once.

* **USER INTERFACE IMPROVEMENTS:** Nothing is radically different, but there’s
been a fair amount of extra spit and polish added, including a convenient
new Dashboard widget that may save you a trip to the Syndication menu,
a lot of effort to make the relationship between global and feed-by-feed
settings more obvious to the user and more easily controllable, to make
navigation between settings pages easier, to sand off a few rough edges,
and to make other improvements on the margins. I hope you’ll like how
it looks.

* **ADDING MULTIPLE FEEDS:** FeedWordPress now provides a convenient mode for
adding multiple feeds at once, using either a copy-and-pasted list, or
else an OPML file. Go to Syndication –> Syndicated Sources and check
out the two new buttons underneath the New Source input box. When you
have to add a number of feeds at once, this can save you considerable
time and trouble.

WITH NAMES WRITTEN IN FOREIGN SCRIPTS:** WordPress 3 is increasingly picky
about what it will accept for new author accounts, and some of the
conditions it imposes can cause error conditions that prevent posts from
being properly syndicated, or properly attributed, if authors happen to
have identical e-mail addresses, or if users are given usernames that are
written in non-Western scripts. FeedWordPress now handles these much
better, and systematically works to avoid clashes between syndicated
authors’ account names or in their e-mail addresses, which should result
in significantly better results in mapping author names to WordPress
user accounts.

In previous versions, the only way for the Categories provided by a
syndicated feed to be mapped into Post Tags was to instruct FWP to
create new tags, rather than new categories, for unfamiliar categories
from the feed. This works fine if you want tags to be the default; but
if you want only a *specific* set of tags, there was no way to get them
without getting most or all other categories imported as tags. You can
now do this by creating a tag (under Posts ==> Post Tags) before
importing the post; when the syndicated category matches a pre-existing
tag, the incoming post will be tagged with that tag, without creating
a local Post Category.

contain inline tags, marked up using the Rel-Tag microformat
, are now tagged with the tags
provided by Rel-Tag format links.

result of building in support for a potentially endless set of custom
taxonomies (see below), but in general there has been a great deal of
effort towards giving you more control over how categories and tags
provided by the feed are mapped into terms on the local blog. In
particular, you can now force FeedWordPress to create only categories
from categories and tags provided by the feed; or to create only tags;
or to search both categories and tags for a match; or you can simply
force it to drop all of the categories provided by the feed and use
only categories or tags that you explicitly provide. In addition, you
can now also choose whether to override global categories settings with
a local, feed-specific setting; or whether to *add together* *both* the
global categories and the local feed-specific categories — depending
on whatever your use-case may demand.

super-geeky, but if you use other plugins or themes that make
significant use of WordPress’s support for custom post types and custom
taxonomies, you may be pleased to find that FeedWordPress now allows you
to feed incoming posts into any custom feed type that you wish, and to
map categories and tags from the feed to custom taxonomies as well as
to the standard Category and Tag taxonomies.

would like to use FeedWordPress’s support for storing custom meta-data
from feed elements in the custom fields for a post (for example, to
store geolocation data or iTunes media meta-data), you’ll find that it’s
now much easier for you to access these namespaced elements. You always
could access them, but in previous versions you might have to write
something ugly like `$(/{http://www.w3.org/2003/01/geo/wgs84_pos#}lat)`
just to get at the value of a `` tag. Now, as long as you use
the same mnemonic codes that the feed producer used, you should always
be able to write a nice, simple expression like `$(/geo:lat)` to get the
value of a `` tag. Huzzah!

* **CUSTOM DIRECTORY STRUCTURE SUPPORT:** if you poke at it enough, WordPress
is relatively flexible about where it should store admin interface code,
uploaded content, plugins, and a number of other things that occupy an
important place in the WordPress directory structure. Previous versions
of FeedWordPress encountered serious errors or broke entirely when used
with directory structures other than the default. This should now be
fixed: FWP now supports custom directory structures wherever WordPress
allows them to be customized, rather than depending on the default
locations. Enjoy your freedom!

have been too many improvements to list them all in this ChangeLog, but
it means that much more power and ease for folks who are customizing
FeedWordPress through PHP filters or add-on modules. Fuller
documentation will be put up at the Wiki at feedwordpress.radgeek.org

and enjoy!(http://downloads.wordpress.org/plugin/feedwordpress.2010.0903.zip) If you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or me a line(/contact) to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you’re using, what version of WordPress you’re using it with, which web browser you are using to view the FeedWordPress user interface, and try to tell me, as clearly as possible,

One of the remarkable fact about this release is the number of the new features in this release — including improved namespace support, subscription to multiple feeds, and custom post type and taxonomy support — that I was able to make due to the generous support of FeedWordPress users, whose gifts made it possible for me to devote a several solid days to some major new conveniences and some significant internal overhauls that have dramatically improved FeedWordPres’s flexibility and power. I’d like to thank all the FWP users who have helped make ongoing development possible. It’s really flattering, y’all, and I’m incredibly grateful that you’ve made it possible for me to devote the needed time to this project.

As always, please remember that your generous gifts to project tip jar(/donate) make ongoing development, quick fixes and timely support for FeedWordPress possible.

Now get out there and enjoy 2010.0903(http://downloads.wordpress.org/plugin/feedwordpress.2010.0903.zip)!

suggestion for the next update.

I always wondered if there existed any easier method to display images with except/thumbnail with the aggregated content using feedwordpress. Unfortunately it has been talk of the web and still no perfect solution. The use of string “snap” is compromising solution.

I would like to suggest and this idea is withdrawn from the pligg module. they had a code that automatically fetches the website thumbnail in rss import module. If a add on as such could be built or could be integrated in core , to automatically add the website snap from snapping services and add that with the excerpt, it would be rocking.
Hope there would be plausible discussion in this regard.

Thanks and Regards.

Custom Post Settings Not Pulling GEORSS Elements from Atom Feed

I can not get FWP to pull a GEORSS element from a atom feed and set a Custom Post Field…

My incoming atom feed looks like this:

33.831032 -117.893439

FWP should allow me to set “Custom Post Settings” which sets “Custom fields can be used to add extra metadata to a post that you can use in themes.”
Syndication -> Post & Links -> Custom Post Settings

My key/value looks like this:
Key: wpgeorsspt
Value: $(georss:point)

But the field never gets created or filled in my posts.

Is this because FWP does not support GEORSS elements?

FeedWordPress 0.993: WordPress 2.5.1 compatibility and a couple new features

Update 2008-11-06: FeedWordPress 0.993 is now out of date. You can download the latest release — 2008.1105 at the time of this writing — from the project homepage.

version 0.993 is now available for download.[1]

[2]: http://wordpress.org/extend/plugins/feedwordpress/feedwordpress.0.993.zip

There are a few new features that I am in the midst of working on for an upcoming release of FeedWordPress, but I have released version 0.993 now in order to resolve critical compatability issue with WordPress 2.5.1[3]. I am still doing compatibility testing to see whether there are any kinks in compatibility with WordPress 2.5.x, but upgrading to this release should eliminate the fatal error that prevented 2.5.1 users from accessing the Syndication Options and the feed settings pages from within the WordPress pages. There are some small bug fixes and the beginning groundwork for some features that will become more fleshed out in the upcoming, more feature-rich release, which aren’t worth going into in detail; besides those, here is what’s new since FeedWordPress 0.992:

* **WORDPRESS 2.5.1 COMPATIBILITY:** FeedWordPress should now be compatible
with WordPress 2.5.1.

* **WORDPRESS 2.5 INTERFACE IMPROVEMENTS:** FeedWordPress’s Dashboard
interface has undergone several cosmetic changes that should help it
integrate better with the WordPress Dashboard interface in WordPress
version 2.5.x.

can now take advantage of WordPress’s new “Pending Review” features for
incoming syndicated posts. Posts marked as “Pending Review” are not
published immediately, but are marked as ready to be reviewed by an
Administrator or Editor, who can then choose to publish the post or
hold it back. If you want to review syndicated posts from a particular
feed, or from all feeds, before they are posted, then use
Syndication –> Syndicated Sites –> Edit or Syndication –> Options to
change the settings for handling new posts.

* **AWARE OF NEW URI FOR del.icio.us FEEDS:** Previous releases of
FeedWordPress already automatically split del.icio.us tags up
appropriately appropriately when generating categories. (del.icio.us
feeds smoosh all the tags into a single `` element,
separated by spaces; FeedWordPress un-smooshes them into multiple
categories by separating them at whitespace.) Unfortunately, del.icio.us
recently broke the existing behavior by changing host names for their
feeds from del.icio.us to feeds.delicious.com. Version 0.993 accounts
for the new host name and un-breaks the tag splitting.

[4]: http://projects.radgeek.com/2008/04/27/feedwordpress-and-wordpress-251-compatability-issue/

**If you have put off upgrading to WordPress 2.5.1 due to this bug, and plan to upgrade after installing FeedWordPress 0.993, please remember** that *after* you upgrade WordPress, will need to reinstall the FeedWordPress MagpieRSS upgrades[5] in order to keep your feed parsing from getting broken.

[6]: http://projects.radgeek.com/2008/04/18/compatability-bugs-and-possible-quick-fixes-for-issues-with-feedwordpress-after-upgrading-to-wordpress-25/

Enjoy! As I mentioned, I’m actively working on a release, probably due sometime before the end of the month, including bug fixes and a few significant new features, so let me know about any ongoing issues that you may still have.

FeedWordPress and WordPress 2.5.1 compatibility issue

I wanted to post a quick note because I have received several reports of a serious compatability issue between FeedWordPress and the most recent public release of WordPress, version 2.5.1. Although FWP is broadly compatible with the initial release of WordPress 2.5, the recent update apparently eliminates an interface function that FeedWordPress used to display a box for selecting categories in the Syndication options and the feed settings pages. Since the function no longer exists in WordPress 2.5.1, it means that a fatal PHP error (shown either by a prematurely cut-off display, or in the form of a printed PHP error message) will be triggered if you attempt to use either of these pages in the WordPress Dashboard. (As far as I know, syndication will continue to work fine with WordPress 2.5.1; the error will prevent you from changing configuration settings from within the WordPress Dashboard.)

This problem should, hopefully, be something that it won’t be too hard to fix once I am able to sit down and work on it. Unfortunately, the release came after I had left to visit family in Alabama, so I can’t get to making the fix until I return home early in the upcoming week. Once I do, I’ll release an immediate compatability fix, and then return to work on a more thorough update, which I hope to release sometime during the month of May.

I’ll check back in with y’all once I’ve gotten home.

Upgrade Downgrade: Compatibility bugs and possible quick fixes for issues with FeedWordPress after upgrading to WordPress 2.5

2.5[1] was recently released, and as a result many FeedWordPress users have upgraded their blogs to the latest version of WordPress. I am currently in the process of testing for any compatability issues between WordPress 2.5 and the development version of FeedWordPress (0.993a); if I notice any definite problems, then I will make them high-priority bug fixes and try to push out the release of 0.993 as quickly as possible. (That probably means either tonight, or some time around the end of the month, depending on when I find any problems that I may find.) If you have tried using FeedWordPress with WordPress 2.5, either in version 0.992 or in the current trunk development version, and have noticed any problems since the upgrade that aren’t fixed by what I’m about to suggest, then please feel free to report them in the comments here or me by e-mail(http://radgeek.com/contact), as you prefer. The most helpful bug reports are those that state, in as much detail as possible, (1) what precisely is going wrong, (2) under what conditions, (3) with what version of FeedWordPress, (4) under what version of PHP, and, if the problem is with syndicating posts, then (5) with which feeds at which specific URIs. If you are getting symptoms of a fatal error (either a printed error message or a blank screen where a page should be), then you can also help me out a lot by copying and pasting the contents of the error message into your report, or, if you have a blank screen, checking the bottom of your web server’s error logs to see if there is a PHP error report down there, and, if so, copying and pasting that.

2.5: http://codex.wordpress.org/Version_2.5

That said, one of the most common sources of error reports when new versions of WordPress are released come not from a real compatability issue, but rather from the fact that, if you’re not careful, upgrading your copy of WordPress will downgrade your copy of MagpieRSS from the newer version shipped with FWP to the very old and busted version that WordPress continues, for whatever reason, to ship with new releases of WordPress.

### Diagnosis ###

Here are the most common symptoms of this problem:

* Some feeds (notably, feeds produced by Blogger and other Atom 1.0 feeds) stop syndicating post contents. You get the headline of the post and nothing else.

  • Some feeds (notably, those produced by blogs hosted at WordPress.com (!)) start appearing with just the capital letter A as the content of the post.
  • Categories stop being properly syndicated. Everything is placed in Uncategorized or in bizarre, mashed-up categories (only one per post) that seem to contain several category names.
  • Podcast attachments are no longer syndicated.

And so on, and so forth. If you notice these problems with your feeds just after you’ve upgraded your copy of WordPress, it’s probably because you need to re-install the MagpieRSS upgrade.

### Cure ###

Here’s how you do that. In the FeedWordPress plugin directory (`wp-content/plugins/feedwordpress/`, relative to your WordPress installation), there is a directory called `MagpieRSS-upgrade`, which contains, or at least at one point contained, two files, `rss.php` and `rss-functions.php`. If you still have these files, you need to copy them to your WordPress `wp-includes` directory, where they will overwrite the older version of MagpieRSS that ships with WordPress. If you do not (because, for example, you moved them rather than copying them when you first installed FeedWordPress), then you can get new copies of these files by downloading the latest version of FeedWordPress, and extracting these two files from the archive.

### Etiology and pognosis for the patient ###

The reason that this happens is that every installation of WordPress includes a *very* old version of MagpieRSS, the library that FeedWordPress uses to parse the feeds that it syndicates. As of the 2.5 release, WordPress *still* ships with package derived from MagpieRSS 0.51(http://trac.wordpress.org/browser/tags/2.5/wp-includes/rss.php), which is the same version shipped with when I started work on FeedWordPress three years ago(http://trac.wordpress.org/browser/tags/1.5/wp-includes/rss-functions.php). This version of MagpieRSS is adequate for what WordPress needs it to do (basically, fetch headlines for the Dashboard from a select few feeds), but it was already outdated three years ago, and it is especially outdated now–it could not handle multiple categories; it could not handle enclosures, it could not translate feeds in alternate encodings; and, importantly, it cannot correctly handle Atom 1.0 feeds (now the default for Blogger feeds) or feeds with MediaRSS extensions (now the default for WordPress.com feeds). Unfortunately, since there *is* a version of MagpieRSS, which is loaded every time you load WordPress, it is hard to drop in a newer version which can do these things without causing errors from the collision in function and class names.

The solution I settled on was the bundled MagpieRSS upgrades, which in the past I sometimes described as optional, but which now really are mandatory if you hope to do any serious syndication in the modern environment. Users can avoid collisions by copying the upgrade so that it just *overwrites* the older version in `wp-includes`. Problem solved for the time being.

But the downside of this solution is that every time an upgrade of WordPress comes out, it comes out with older versions of the MagpieRSS package included, and when you overwrite all the files in wp-includes with the newer files from the WordPress release, one of the things you overwrite is your upgraded copy of `rss.php`. Meaning that, unless you remember to re-upgrade MagpieRSS every time you upgrade WordPress (something which is easy enough to forget), it breaks your syndication until you remember, or I remind you, to re-do the upgrade.

I frankly consider this a design flaw in FeedWordPress, but it’s not a flaw that is easy for me to fix. I am considering different ways of getting around it, and honestly the most likely solution at this point is probably simply to abandon MagpieRSS and package another feed parsing package (such as [2][3]) in the `feedwordpress` plugin directory, where upgrades to the WordPress core code cannot interfere with it. But doing that will involve pretty dramatically refactoring some of FeedWordPress’s internal workings, and that may take a while. In the meantime, if you have a working aggregator, you should probably apply this quick fix and see how many of your problems it solves.

[4]: http://simplepie.org/

FeedWordPress 0.992: author remapping and URI bug fix

Update 2008-11-06: FeedWordPress 0.992 is now out of date. You can download the latest release — 2008.1105 at the time of this writing — from the project homepage.

0.992 is now available for download.(http://wordpress.org/extend/plugins/feedwordpress/feedwordpress.0.992.zip)

Since I’ve had to spend time either traveling or working on other projects, it’s been longer than I would have liked since the last update of FeedWordPress. This release is a rather limited one: it fixes one outstanding bug and adds one major new feature:

WordPress 2.x’s handling of URIs in Blogroll links created problems for
updating any feeds whose URIs included an ampersand character, such as
Google News RSS feeds and other feeds that have multiple parameters
passed through HTTP GET. If you experienced this bug, the most likely
effect was that FeedWordPress simply would not import new posts from a
feed when instructred to do so, returning a “0 new posts” response. In
other cases, it might lead to unpredictable results from feed updates,
such as importing posts which were not contained in the feed being
syndicated, but which did appear elsewhere on the same website. This bug
has, hopefully, been resolved, by correcting for the bug in WordPress.

2. **NEW FEATURE – AUTHOR RE-MAPPING:** I’ve been promising this one to my e-mail
correspondents for a couple of months now; and I’m happy to announce that I’ve been able to polish
up the preliminary implementation that I was working on late last year and make it ready for general
consumption. FeedWordPress now offers a new feature in the site-wide Syndication Options
(Syndication –> Options), and in the settings for each syndicated feed (Syndication –> Syndicated
Sites –> Edit). You can now create re-mapping rules to determine how author names in a feed are
translated into usernames within the WordPress database.

Traditionally, what FeedWordPress has done, when adding a post, is to take the author information
from the feed and search the WordPress database to determine whether there is a username with
the same name or e-mail address. If it found a match, then the new post would be assigned to that
user. If there was no match, then FeedWordPress would consult the settings for the feed, or the
global default settings, for what to do with an unfamiliar author. These could be set by the
administrator from within the WordPress Dashbard, with three options: (1) create a new user
account with the same name as the author, and assign the new post to that new user; (2) filter out
the post rather than adding it to the database; or (3) assign the new post to a default user (user #1
in the WordPress database, i.e. the site administrator).

With the new re-mapping feature, you have considerably more control over what FeedWordPress
does with post authorship data from the feed. In the settings for each syndicated feed (Syndication
–> Syndicated Sites –> Edit), you can now define rules that tell FeedWordPress what to do with
each particular author name on that feed — to filter out all posts by that author, or to assign posts
by that author to any username that you like. You can also tell FeedWordPress what to do with
usernames that aren’t listed in your rules, and which don’t match any of the users already in the
WordPress database — to filter out the posts, or to assign them to any username you like, or to
create a new username to match the new author, or to follow the default setting for new author
names on all feeds. The site-wide default setting can be changed using Options –> Syndication.

So, for example, one of my friends runs a FeedWordPress aggregator site that syndicates
. The problem is that, like an increasing number of WordPress
blogs out there, all the posts on this blog are attributed to Administrator. For those visiting
the blog directly, it’s clear that Administrator is the blogger; but when someone aggregates
the blog on another website, the naming now mistakenly implies that the Administrator of the
*aggregator* website is the author of the post. (It also means that if two or more blogs both
attribute their posts to Administrator, the aggregator site will mistakenly treat all of the
posts from all of those blogs as being by the same author.) With the new re-mapping feature, you
could now define a rule that would attribute posts by Administrator from the praxeology.net
feed to a different username — in this case, Roderick Long. Problem solved. Huzzah!

One more note before I go. I regret that I haven’t been able to develop FeedWordPress more actively than I currently am developing it, or to spend as much time handling and responding to bug reports as I would like. I originally created FeedWordPress for my own use, and made it available to others in the hope that it would be useful, so the best guarantee of getting a feature added or a bug identified and quickly fixed has always been whether it’s one that I personally encountered, or one of my friends encountered, in the course of using it. But I’m really very pleased with how much uptake and interest there has been for FeedWordPress. Ideally I would like to devote much more time to FeedWordPress development and support than I currently do. But I earn my living freelance, by the hour or by the project, and I do have to pay my rent every month, so the only way that I can keep up with this on more than a limited and casual basis is if the donations from FeedWordPress users allow me to free up the time needed to work on FeedWordPress, rather than spending the same time looking for paying gigs.

So, if you would like to see more regular upgrades and bugfixes, and more rapid replies to your support questions, I’d urge you to consider how much ongoing development and support for FWP is worth to you, and consider making a contribution through the project donation jar at . It’s in your hands, and anything you can offer will help. (If I had $5 for everyone who ever sent me a FeedWordPress tech support question, I’d be flush for the next several years….)


How to deal with dates and timestamps in WordPress

If I hadn’t moved across the country recently, I probably never would have discovered a bug in how FeedWordPress handles the timestamps of syndicated posts. Fortunately, I did, and this bug is fixed in recently-released version 0.99[1].

[2]: http://projects.radgeek.com/2007/09/24/feedwordpress-099-is-hereby-released-enjoy-wordpress-22-and-23-compatability-bug-fixes-major-new-features-and-updates-without-cron/

To explain what I mean, I’ll have to back up a bit, first.

There are a couple things that FeedWordPress uses date and time information for:

1. **Dating new posts:** When a new post comes in over a feed, FeedWordPress uses the date and time information reported by feeds to date posts appropriately in the WordPress database. This means parsing date and time information and putting the resulting timestamps into the database.

2. **Checking for updates to existing posts:** After a post has been imported into the WordPress database, FeedWordPress will keep checking to see whether the feed reports any updates to that post. This means comparing the last-updated timestamp on the feed to the last-updated timestamp in the database to see which version is newer.

The problem that I noticed came about because FeedWordPress was trying to do something sensible and easy to handle *time zones* when it was doing all this. Date/time handling in WordPress is fairly easy, *once you understand what to do*, but it is certainly anything but sensible, and therein lies the problem.

Part of the problem is, as usual, the stack of programs on top of which FeedWordPress has to sit. In order to get dates from the feed, FeedWordPress has to process two different human-readable date formats. RSS feeds use 822(http://www.faqs.org/rfcs/rfc822.html) (or something like it), which FeedWordPress can convert to a Unix timestamp using the PHP [3](http://us.php.net/strtotime) function. Atom feeds use the DateTime Format(http://www.w3.org/TR/NOTE-datetime), which FeedWordPress can convert to a Unix timestamp using a custom function provided by MagpieRSS, called `parse_w3cdtf()`. Unix timestamps supposedly do not vary by time zone: a Unix timestamp for a particular time is defined as the number of seconds between that time and 12:00 midnight on January 1, 1970 *Greenwich Mean Time*. So both `strtotime()` and `parse_w3cdtf()` make use of the time zone data provided by the format that they handle, and use it to convert the date-time information to a timestamp based on GMT.

So far so good. Now, when converting a syndicated item into a post for the WordPress database, FeedWordPress has to generate *four* different timestamps: `post_date`, which gives the date and time that the post was first published *in the local time zone*; `post_date_gmt`, which gives the date and time that the post was first published *in Greenwich Mean Time*; `post_modified`, which gives the *local* date and time that the post was last modified; and `post_modified_gmt`, which gives the *GMT* date and time that the post was last modified. These fields are stored in the WordPress database as DATETIME objects(http://dev.mysql.com/doc/refman/5.0/en/datetime.html). WordPress later converts them back into Unix timestamps whenever it is necessary for formatting purposes. Here is how I did this in version 0.981:

$post[4] = date(‘Y-m-d H:i:s’,
? $post[7][8]
: $post[9][10]));
$post[11] = date(‘Y-m-d H:i:s’,
$post[14] = gmdate(‘Y-m-d H:i:s’,
? $post[17][18]
: $post[19][20]));
$post[21] = gmdate(‘Y-m-d H:i:s’,

The `$post[24]` array contains the Unix timestamps that FeedWordPress got from the feed. The PHP [25](http://us3.php.net/manual/en/function.date.php) function formats a Unix timestamp relative to the local time zone. The [26](http://us3.php.net/manual/en/function.gmdate.php) function formats it relative to Greenwich Mean Time. Given that I need local and Greenwich representations of the date and time that the post was first published, and of the date and time that the post was last updated, this seems like the sensible thing to do. But in spite of (or because of) its seeming so sensible, this way of generating post timestamps introduces a subtle bug. More on that later.

When determining whether or not a post has been updated, FeedWordPress uses two items of information: the last-updated date and time provided by the feed, and the last-updated date and time stored in the database. In order to get these two dates into a common format so that they can be compared, FeedWordPress uses either `strtotime()` or `parse_w3cdtf()` to convert the feed’s last-updated date and time to a Unix timestamp, and it uses the MySQL function [27](http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html#function_unix-timestamp) to get a Unix timestamp from the last-updated date and time in the database. So here is how I did this in version 0.981:

$guid = $post[28];
$result = $wpdb->get_row(”
SELECT id, guid, UNIX_TIMESTAMP(post_modified) AS modified
FROM $wpdb->posts WHERE guid=’$guid’

if (!$result) :
$freshness = 2; // New content
elseif ($post[29][30] > $result->modified) :
$freshness = 1; // Updated content
else :
$freshness = 0;

`strtotime()` and `parse_w3cdtf()` take account of time zone data; `UNIX_TIMESTAMP()` presumes that the time is in the local time zone. So, I just need to feed it the DATETIME object that’s in the local time zone — `post_modified` instead of `post_modified_gmt`, right? Wrong. Again, in spite of (or because of) its seeming so sensible, this way of comparing timestamps introduces a subtle bug.

Here’s what was wrong with both of these sensible steps: **in any given WordPress installation, there are *three* potentially distinct local time zones that you have to consider**:

1. The local time zone of the web server on which WordPress is running (usually set by the web server’s administrator);

2. The local time zone of the MySQL server that provides the WordPress database (usually set by the MySQL server’s administrator);

3. The local time zone set by the user in WordPress’s General Options page (set by the blog owner);

If any of these differ from each other, then the mismatch could cause problems for the way that older versions of FeedWordPress handled dates.

When the PHP date and time functions convert back and forth between human-readable formats and Unix time stamps, they do so relative to the web server’s local time zone. When MySQL converts a DATETIME object to a Unix timestamp, it does so relative to the MySQL server’s local time zone. When WordPress prepares dates and times for storage in the database, or processes them for sorting and displaying posts, it does so relative to WordPress’s local time zone, as set under General Options.

So, when FeedWordPress used the PHP date and time functions to generate `post_date` and `post_modified`, it used *the wrong time zone* — WordPress expects these to be local times *in the time zone set under General Options*, but the PHP functions use the *web server’s* local time zone. If the user has set a different time zone from the web server’s default time zone, then this date and time information will be incorrect. In order to get the *correct* time, we need to get the time zone information from the WordPress database, and then manually apply that offset, rather than leaning on PHP’s date and time functions. So here is how we do this task in FeedWordPress 0.99:

// Dealing with timestamps in WordPress is so fucking fucked.
$offset = (int) get_option(‘gmt_offset’) * 60 * 60;
$this->post[31] =
gmdate(‘Y-m-d H:i:s’, $this->published() + $offset);
$this->post[32] =
gmdate(‘Y-m-d H:i:s’, $this->updated() + $offset);
$this->post[33] =
gmdate(‘Y-m-d H:i:s’, $this->published());
$this->post[34] =
gmdate(‘Y-m-d H:i:s’, $this->updated());

Note that we use `gmdate()` in all cases, because we are doing the time zone offset manually rather than letting PHP do it for us.

On the other hand, when FeedWordPress used the MySQL date and time functions to convert MySQL DATETIME objects to Unix timestamps, it used the wrong time zone again — since it was using `post_modified`, and in the older versions of FeedWordPress `post_modified` was generated using PHP date and time functions, the time was a local time relative to the *web server’s* time zone. But `UNIX_TIMESTAMP()` presumes that the time it is given is a local time relative to the *MySQL server’s* time zone. Usually this difference should cause no problem, since the web server and the MySQL server are the same machine, or if not the same machine, at least machines that are located in the same place as one another. But if that assumption ever fails, `UNIX_TIMESTAMP()` will return the wrong time–a time that is either a few hours before, or a few hours after, the real last-updated time. Which will mean that posts either get updated when there’s nothing new to update, or don’t get updated even when there is something new to include.

So we need to convert the MySQL DATETIME value to a Unix timestamp in a context where we know, and can set, the right time zone for the conversion. In this case, the best thing to do is to get the *GMT* date and time of the last update (in order to avoid issues that might arise from changes in the local timezone setting in WordPress), and then manually convert that into a Unix timestamp using a function that works relative to GMT. So here’s how FeedWordPress 0.99 checks the update times against each other:

$guid = $wpdb->escape($this->guid());

$result = $wpdb->get_row(”
SELECT id, guid, post_modified_gmt
FROM $wpdb->posts WHERE guid=’$guid’

preg_match(‘/([35]+)-([36]+)-([37]+) ([38]+):([39]+):([40]+)/’,
$result->post_modified_gmt, $backref);
$updated = gmmktime($backref[41], $backref[42], $backref[43],
$backref[44], $backref[45], $backref[46]);
if (!$result) :
$this->_freshness = 2; // New content
elseif ($this->updated() > $updated) :
$this->_freshness = 1; // Updated content
$this->_wp_id = $result->id;
else :
$this->_freshness = 0; // Same old, same old
$this->_wp_id = $result->id;

`post_modified_gmt` always returns the string representation of a MySQL DATETIME object; the regular expression breaks that representation down into its component parts; and the PHP function [47](http://us3.php.net/gmmktime) reassembles those parts into a Unix timestamp. (You might worry that using the PHP date and time functions might reintroduce the first problem, since it doesn’t account for the local time zone set in WordPress. But since everything is guaranteed to be in GMT, this problem doesn’t arise.)

Strictly speaking, it would probably be better to use MySQL functions, instead of a regular expression, to extract the parts of the MySQL DATETIME object, since ostensibly MySQL knows more about its internal formats than PHP does. In practice this is not likely to make a difference, but it’s likely that in future releases of FeedWordPress I’ll change the section to something more like this:

$guid = $wpdb->escape($this->guid());

$result = $wpdb->get_row(”
id, guid,
YEAR(post_modified_gmt) AS year,
MONTH(post_modified_gmt) AS month,
DAYOFMONTH(post_modified_gmt) AS day,
HOUR(post_modified_gmt) AS hour,
MINUTE(post_modified_gmt) AS minute,
SECOND(post_modified_gmt) AS second
FROM $wpdb->posts WHERE guid=’$guid’

if (!$result) :
$this->_freshness = 2; // New content
$updated = gmmktime(
$result->hour, $result->minute, $result->second,
$result->month, $result->day, $result->year
if ($this->updated() > $updated) :
$this->_freshness = 1; // Updated content
$this->_wp_id = $result->id;
else :
$this->_freshness = 0; // Same old, same old
$this->_wp_id = $result->id;

Oh, and in case you were wondering, the reason that moving across the country helped me find this out is that I moved from Eastern Time to Pacific Time, and I changed the default time zone on one of my web servers before I changed the default time zone on my MySQL server. The mismatch exposed this bug while I was doing testing for most recent release of FeedWordPress[48]. Not particularly interesting, but it did expose a bug to fix, and I hope the guide to time zone issues that resulted may be of some interest.