Feed URL Validation Error

Archived
1

I’ve seen this question asked on this forum before without much feedback. I’ve converted my MRSS feed into a JSON feed using a Powershell script I wrote. I tested the resulting JSON file using one of the recommended validaters and it came up clean. I upload the JSON file to my Amazon S3 bucket and tried to reference it from the Feed URL page on the developer’s portal. When I try to save the Feed URL, I get:

The feed URL(s) could not be validated.
The server returned an unsupported content type. Was expecting ‘application/rss+xml’, ‘text/xml’, ‘application/xml’ or ‘application/json’ but received ‘binary/octet-stream’.
At first, I thought it might have something to do with the file extension, so, I changed it from “.json” to “.rss”. That did not work. Then, I tried playing around with the CORS stuff for the bucket and that didn’t help either.

  1. Can I host my json feed with an Amazon S3 bucket?

  2. What is the correct file extension for the feed file? .rss?

2

The extension has nothing to do with anything. However, some web hosting providers are smart enough to assume that a “.json” file applies to a Content-Type of application/json, and serve the content with that Content-Type.

What’s important is the HTTP Content-Type header returned by the server. It appears that Direct Publisher is looking for a specific Content-Type header of application/json, for example.

I don’t know anything about Amazon S3, but when researching this problem trying search for “Content-Type” http header (sometimes also referred to as a mime-type). Look for a way you can specify a specific Content-Type for a particular file or file extension, either from a control panel or settings value, or some other url that will serve the same content with a different Content-Type. You’d probably get better help on StackOverflow under the ‘amazon-web-services’ and ‘amazon-s3’ tags.

3

belltown,

Thanks for the reply. It was just enough info to point me in the right direction. I figured out the problem and have corrected it. Here’s the jist.

When uploading a JSON file to Amazon S3, the content type does not default to “json”. I had to add that as a bit of meta data as part of the upload. Their interface allows you to make changes to the meta data as part of the upload.

Unfortunately, that has not fully resolved my problem. I now get a cryptic message:

The feed URL(s) could not be validated.
JSON (SyntaxError: Unexpected token � in JSON at position 0)
from the Roku portal when trying to validate the feed file. And, as I stated, the file validates fine through the online validaters.

4

It looks like you’re making progress. If you or the Roku DP uploader is seeing a strange-looking character or two at the start of the file, then you need to check the file’s encoding.

I would imagine that DP expects the Json files to be encoded in UTF-8 without a Byte-Order Mark (BOM). It’s possible you have a BOM at the start of the file. That may be a result of the way the file was encoded before you uploaded it (especially if you’re on a Windows computer using certain text editors), or as a result of the way AWS is choosing to serve the file. First, what OS and text editor are you using to generate your .json file. Second, is there anywhere in your AWS control panel that you can specify an encoding (sometimes called a “charset”) for the file? If you can specify an encoding somewhere in the AWS control panel/meta-data, etc., then set it to “UTF-8”.

5

On a similar note, I keep getting this error message:

The feed URL(s) could not be validated.

  • There was a transport-level error - (Unknown ERROR).

    I’m no JSON coding expert, but the JSON formatter says my code is valid. Any idea how to remedy (or links to other threads that might assist)?
    Thanks. :slightly_smiling_face:

6

“Baradanikto” wrote:
belltown,

Thanks for the reply. It was just enough info to point me in the right direction. I figured out the problem and have corrected it. Here’s the jist.

When uploading a JSON file to Amazon S3, the content type does not default to “json”. I had to add that as a bit of meta data as part of the upload. Their interface allows you to make changes to the meta data as part of the upload.

Unfortunately, that has not fully resolved my problem. I now get a cryptic message:

The feed URL(s) could not be validated.
JSON (SyntaxError: Unexpected token � in JSON at position 0)
from the Roku portal when trying to validate the feed file. And, as I stated, the file validates fine through the online validaters.
Just a thought to help troubleshoot. Try hosting the file at myjson.com and see if it passes the Roku checklist validator. That will at least let you know if its a file error or Amazon S3 specific. Secondly, some people mention the Header section of the feed? Try checking the sample Roku feed and just paste that into yours again (provider name, language, last updated etc).
Just a thought.

7

I’ve made some more progress. I was able to upload the json file specifying a different content-type of “application/json”. However, this only helped a little. I still get the “unexpected token” error message “at position 0”, but, it no longer has a funny character in the message. I’ll assume that the content-type correction helped that.

I took some advice and used myjson.com to try my feed and it spits back I have “no valid items in my feed”. And, "27 items have error: Required shortFormVideo content is missing." I’ve validated the syntax using the online services. Is there a specific schema just for Roku validation? Pasnow makes reference to a “Roku checklist validator”. Where is that? Is that one of the online services that Roku recommends to check your syntax? BTW…my feed does include “shortFormVideo” tags and “content” tags. The only potential discrepancy I see is the order of my json tags. Does anyone know if the order of the tags is significant? For example, my header contains a “providerName”, but, it’s not the first tag. I would hope the order of the tags is NOT significant.

Lastly, I posted on the Amazon S3 forum because I may have found that they will not serve json files. It appears they may only server xml. I’ve asked for someone to clarify that. I’ll report back on that when I have an answer.

Oh, I forgot to mention, I wrote a powershell script that converts my xml feed to json. It runs on Windows 7.

8

To remove problematic characters in your json file tun this regex to remove them, then save the result –
in php –
$json=json_decode(preg_replace(‘/[\x00-\x1F\x80-\xFF]/’,‘’,$source),true);

Where $source is the contents of your input file.

9

Having same error as Baradanikto. Validator says JSON code is valid, but Roku direct-publisher repeats error message: “No valid items in the feed.”
Very discouraging after spending lengthy amounts of time trying to get it right. Stuck like he is–any thoughts appreciated.

10

Does it give anything below it? Like “No valid images in 1 feed” or something? Also check to see if the top header content begins similar to the Roku sample feed. These are just guessing but maybe double thru some of the semantics. See if a paste into a myjson.com url works? Could it be a file format issue?

11

I think I may have found the cause of my error. But, I think I’m going to need someone from Roku to explain this. My Powershell script converts my MRSS xml feed to a JSON feed. In the process, it converts the “Content” object of the shortFormVideo object as a regular key/value pair which results in a “[” in the resulting JSON. Looking at the feed specification, I noticed that the content object does not seem to conform to the specs like the other “objects”. It actually uses a “{”. So, while my code looks like:

"content": [
                {

The feed specification shows it as:

"content": { 

with key/value pairs. I don’t quite understand the discrepancy and why the “content” object does not behave syntactically like the other objects. And, the reason this is a problem for me, and, possibly others, is that I can use a Powershell command called ConvertTo-Json that allows me to automate the creation of the Json file from key/value pairs. This discrepancy breaks that command. I’m currently in the process of validating my suspicions, but, I’m stuck in “The service is still processing your feed”.

I’d love it if someone from Roku could comment on this. I’d like not to have to create the Json by hand.

12

You might want to study up on JSON some more. Start with http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf.

In JSON, an object is ALWAYS enclosed in braces { }. An array is ALWAYS enclosed in square brackets .

If your conversion is putting square brackets around an item, it probably thinks it’s made up of a list (array) of items.

13

… and don’t forget to increase the -Depth parameter in Powershell’s ConvertTo-Json cmdlet.

14

With some prodding from belltown, I was able to determine that my Powershell script had a bug that converted the Content object incorrectly. I’ve fixed it and my json now validates correctly and the channel displays properly.

I’m still having problems determining where to host the json. In the interim, for testing, I’m using myjason.com. I’m just not sure about how secure that site is.

15

“Baradanikto” wrote:
I’m still having problems determining where to host the json. In the interim, for testing, I’m using myjason.com. I’m just not sure about how secure that site is.

I don’t understand why so many people are having so much trouble finding a place to put their JSON files. Is it that you don’t want to pay anything?

There are thousands of low-cost, shared web-hosting sites out there where you can put your JSON files, web sites, and anything else you want. Just Google it. There are companies like HostGator that will charge around $6/mo (for a 12-month plan), and offer a free money-back 45-day guarantee.

If you don’t want to pay anything, there are other sites that will let you host stuff for free, just with limitations. For example, Gigarank.net does a fairly good job. Their only requirement is that you make 10 forum posts. After that you can host your site without any more posts.

Then there are the big-time Cloud service providers. Both Amazon Web Services and Google Cloud Platform provide a free tier for the first 12 months. After that, the costs are minimal for something as low-use as hosting a DP JSON feed.

If you don’t want to pay anything at all, like with many things these days, your options become more limited and more kludgy.

16

belltown…Please don’t make assumptions. Take a moment to re-read the first part of this thread where I indicate that I am trying to serve the json file from my Amazon S3 bucket, which, by the way, is a paid service. You provided some initial help, but, I’m still not able to serve the json file from the S3 bucket. I have a post on Amazon asking for help, but, it looks like they may only serve xml from their buckets. I’ve also tried to serve the file from my channel’s associated web site, but, I can’t get the content type correct when I try to ftp it. So, far, the only place that seems to serve the json correctly is sites like myjson.

I’d love to hear your solution to where and how to serve the json file.

17

Sorry, I wasn’t meaning to be assumptuous. I’m just trying to help.

I’m just used to using hosting providers that let you host whatever kind of files you want on their servers (within reason). I’ve used a couple of dozen such services over the years. They all work very much the same way and will almost certainly allow you to store your json files. I gave you a couple of examples, one that’s known to be fairly reliable, but you have to pay for (although there’s the 45-day money-back guarantee), and the other that’s free, which I’ve been using for several years since they started, with only one major issue that whole time.

Amazon S3 sounds like it should be a good solution for what you want to do, especially if you’re already using it and paying for it.

I thought you were on track with getting your feed working on S3. Initially, the file was not being served with the correct Content-Type header, but you said you figured that part out. Then you said you were having further problems with strange characters at the start of the file. You never responded to the questions I asked you to try and diagnose the problem regarding how you were creating the file and whether there’s anywhere you can specify the character encoding.

Then you said you couldn’t get your feed working even if you hosted it on myjson.com, which would indicate the problem wasn’t with S3 at all, but with the feed. Then you said you found errors related to “the content object does not seem to conform to the specs”, but which were actually a bug in the script you wrote to generate the Json file. Again, problems with your feed, not with your host. Now you say you have another web site associated with your channel. That would seem to be a logical place to host your feed. Yet you say, “I can’t get the content type correct when I try to ftp it”. Well, you don’t normally set the Content-Type when you ftp to server. You configure the server to set the Content-Type when it serves particular file types to the client. S3 is a little different because of the way it sets up its “buckets”.

Basically, you’re going round in circles here. I would go back to square one if I were you and start working with a known, valid, JSON feed file, so you can rule out any problems with the feed itself or its encoding. My recommendation would be to set up a test private DP channel by following the Roku tutorial step-by-step - exactly. Use the feed file they supply and put it somewhere (start with myjson.com, then try your own web site, then try S3). That feed is known to work so if you get the same errors as you’re getting now, then we’ve eliminated the feed as the problem, and can narrow it down to what’s going on with your hosting.

18

belltown…I apologize if I wasn’t clear. But, if you take a look back at my Friday post, you’ll see that with your help, I was able to determine that my json feed syntax problem was due to a bug in my Powershell script. I’ve fixed it and the feed validates correctly if I host it on myjson.com. However, it will not validate from Amazon S3, my web site, or, other services I’ve tried. All of these services seem to serve the json file as either an xml file, or, text/generic. Amazon allows me to change the content-type when uploading, but, it doesn’t seem to make a difference when validating from the Roku developer portal. I’ve tried several FTP products in an effort to upload the correct content-type, but, to no avail. If Roku is expecting application/json when it retrieves the feed, I haven’t found a way or a site that will allow me to serve the file correctly. The only one that seems to work for me right now is myjson.com. There may be other similar sites, but, I’d love to host the file correctly from my own site.

Thanks again for the help you’ve provided to this point.

19

I took Roku’s DP tutorial JSON feed and uploaded it to my web site on Gigarank, to a web site on GitHub, to myjson.com, and to an S3 bucket I just created.

In all cases, the feed is served correctly with the correct Content-Type header, and passes the DP url validation checks, and is successfully uploaded to DP (although it took an hour or two for “processing”).

As far as S3 goes, even though I’ve never used it until a few minutes ago, I ran into no issues at all creating the S3 bucket, and getting the JSON file uploaded and then downloaded with the correct Content-Type.

I’m convinced your problem is with your feed “file”, not with the hosting providers (although you have a lot of misunderstandings of how that stuff all works). When you create a feed on myjson.com, you’re not uploading a “file”, you’re pasting copied text into a web form, which creates its own file with whatever character encoding, byte-order marks, and line-endings it needs. If you upload an actual file (to your web site or S3), you could be uploading something completely different from what DP expects (e.g. a UTF-16, or UTF-8 with BOM, file created using some kind of Windows text editor), which may not work no matter what host you serve it from, and what Content-Type you use – which is why I’ve asked you (twice now) about how you are creating the file, and how it’s encoded.

I’ll repeat the advice I gave at the end of my last post: get a known JSON file (the one Roku provides), and get that working first! Once you’ve figured out how to upload a known, good file to your server (and like I said before, the Content-Type there has nothing to do with FTP), and/or to an S3 bucket (setting the Content-Type meta-data and permissions), then you can figure out how that file differs from the one of your own you’ve been using.

Good luck!

20

The web server you put the json file on HAS to be configured to properly create the headers when serving a json file.. I don’t know how to do this with any server other then IIS, as that is what I run, I’m sure it’s about the same for a linux box and if you rely on someone else to do the dirty work for you like a hosting service, then simply tell them to create the json mime type..

For IIS:

  1. Open IIS Manager

  2. Display properties for the IIS Server

  3. Click MIME Types and then add the JSON extension:

    • File name extension: .json

    • MIME type: application/json

  4. Go back to the properties for IIS Server

  5. Click on Handler Mappings

    • Add a script map

    • Request path: *.json

    • Executable: C:\WINDOWS\system32\inetsrv\asp.dll

    • Name: JSONAnd this works on all IIS 7+ servers including windows xp - 10 pro/enterprise and all flavors of server since server 2000… might work on win 95 and win 98 but really if your running that to serve videos …