How to create a feed?

This article will explain how to create a feed.

Abstract 

Feeditor doesn’t have hard or harsh requirements as to restricting or imposing feed formats or structure. The recommended way though, is a simple two dimensional csv.

There are 3 major formats: delimited (.csv, tsv, etc….), json & xml.

Choose the delimited option when possible. Second choice would be json. Third, as a last resort - xml.

General requirements

If you create your feed on your server as a result of a long running query/routine:

  1. consider caching the result as a static file (valid for 1 hour) and serving the saved cached file (rather than a live query)
  2. For large feeds (query/routine results) - consider compression - your server might have default support for compression of text content. Either implement text compression (on your server), or create an archive, such as .tar.gz, .zip, etc...
  3. Consider streaming the content (while generating the rest) if compression is not implementable 
  4. Encode your content the right way - you should encode the bytes of the query/routine result correctly. Avoid lower/invisible bytes (like bell, etc...)

Delimited (.csv, tsv)

  1. When possible, include column headers as the first row. Try not to use spaces, commas or non-ascii characters in the column headers. If the column headers answer this sample regex, you should have less issues ^[a-zA-z_-]+$
  2. Delimiter - if you are generating the delimited content manually - use TAB as the delimiter - it’s least prone to errors. Otherwise, it’s recommended to use a library/tool provided by the programming platform you are working with. E.g. for python, one example would be pandas DataFrame.to_csv. If you are using a library like pandas - choose either comma (default in pandas) or tab as the output separator. For a manual feed generation + non-standard delimiter (such as ;) - make sure that the number of delimiters is equal across all rows, ensure that empty columns have still padding left (e.g. ;;;) and the delimiter (;) is escaped within regular text (such as title or description).

Json

Use a tool in your programming environment to generate valid json. The structure could be either of those:

  1. [{column1: ‘...’, column2: ‘...’}, {column1: ‘...’, column2: ‘...’}]

or

  1. {‘data’: [{column1: ‘...’, column2: ‘...’}, {column1: ‘...’, column2: ‘...’}]}

Xml 

  1. Reserved xml entities should be escaped.
  2. It’s not enough to just include the correct encoding in the xml declaration - in addition - correct encoding of bytes is crucial.
  3. Avoid complex nested representation of relational databases.
  4. Avoid complex namespace scenarios.
  5. A link on reserved xml entities (that should be escaped), that might make life easier: 

https://www.advancedinstaller.com/user-guide/xml-escaped-chars.html