Get a Working Date/Time Format Using a Tableua Web Data Connector

Here’s a tip that resulted from quite a bit of frustration. It has to do with Tableau’s new-ish Web Data Connector and a difference in JavaScript date functions across browsers (particularly, Chrome and Safari). While trying to pull in JIRA data, depending on which JavaScript method I attempted, I would see either a NULL or NaN (Not a Number) in any date field. The frustrating things is that it worked perfectly in the simulator. I only ran into the problem in Tableau Desktop (on a Mac) and could not find any errors in the logs or console.

To start with, Tableau has a pretty good tutorial to help people get started writing a WDC, which I recommend you check out first if you haven’t already. Within their documentation, you can find a cheat sheet for accepted date/time formats. However, even though I referred to this cheat sheet and tried several formats, I could never get anything to produce a result other than NULL or NaN in Tableau. Again, despite the dates rendering properly using the simulator in Chrome. Writing a WDC wasn’t a high priority, so I would bang my head on it for a while, set it aside for something else, and then come back the next week to do the same thing all over again.

A breakthrough finally came when I asked myself, “Does the simulator behave the same in all browsers?” I fired it up in Firefox and saw the same valid results as I saw in Chrome. Next, I brought up the simulator in Safari. Much to my surprise, the dates produced a NULL or NaN! For my purposes, the WDC was connecting to a proxy server, which then connected to JIRA using PHP’s cUrl functions. In order to confirm my suspicion that Tableau was using Safari, I checked the access log for this proxy server. What user agent did I find when trying to connect using Tableau Desktop, rather than the simulator?

Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X)+AppleWebKit/538.1+(KHTML,+like+Gecko)+Qt/5.4.1+Safari/538.1

As you can see, Tableau was using Safari. A Google search lead me to this question on Stack Overflow, which made me think Safari was not quite comfortable with the date/time string provided by JIRA (e.g.:2016-07-23T05:31:15.000-0400). The accepted answer suggests trying out moment.js, which I did and found it incredibly easy to use. Just remember to include the moment.js file when you add your WDC to the server.

With moment.js in hand, the solution was simple. I included moment.js in my WDC with something like the following:


<script src="moment.js"></script>

Then, I used the following code to get the date string supplied by JIRA into a format supported by Tableau:


var dateFormat = "Y-MM-DD HH:mm:ss";

var createdDate = moment(issues[ii].fields.created).format(dateFormat);

Tableau magically rendered a real date! So, if you find yourself trying to solve the same puzzle, give moment.js a try and see if that doesn’t get you onto the next hurdle (which is getting Tableau Server to a version that is compatible with the WDC found in the tutorial/simulator).

routeToS3: Storing Messages in S3 via Lambda and the API Gateway

(If you want to cut to the chase, skip the first four paragraphs.)

Over the past year, Amazon Web Services (AWS) has previewed and released several new services that have the potential to drive the cost of IT down. This includes services like EFS and Aurora, but the service I was most excited about was Lambda. Lambda is a service that executes code on-demand so you don’t have to pay for an entire EC2 instance to sit around waiting for events. I recall at my previous position having a server that only existed to execute scheduled tasks. As supported languages expand, Lambda has the potential to completely replace such utility servers.

There are many ways to trigger Lambda functions, including S3 events, SNS messages and schedules. But, until recently, it wasn’t straightforward to trigger a Lambda event from outside your AWS environment. Enter Amazon’s fairly new API Gateway. The API Gateway is a super simple way to setup http endpoints that communicate with AWS resources, including Lambda functions. And, you don’t have to be a seasoned developer to use it. In fact, I had only recently started learning some standard concepts while playing around with the Slim Framework for PHP. While understanding RESTful APIs will help the API Gateway feel more natural, you can get started without knowing everything.

Let me back up a bit and explain why I came across the API Gateway in the first place. SendGrid has become our go-to service for sending email from various applications. I can’t say enough good about SendGrid, but it has some intentional limitations. One of those is that it will store no more than 500 events or 7 days (whichever comes first) at a time. You still get all your stats, but if you need to look up what happened to a specific email two weeks ago (or two minutes ago depending on your volume), you’re out of luck. Fortunately, SendGrid thought this through and made an event webhook available that will POST these events as a JSON object to any URL you give it. “Perfect!” I thought, “We can build something to store it in RDS.” But first, I thought it prudent to explore the Internet for pre-built solutions.

My research brought me to Keen.io, which was the only out-of-the-box solution I found that would readily accept and store SendGrid events. If you are here for the exact same solution that I was looking for, I strong recommend checking out Keen.io. The interface is a little slow, but the features and price are right. We would have gone this route in a heartbeat, but had some requirements that the terms of service could not satisfy. With that option gone, I was back to the drawing board. After brainstorming many times with my teammates, we finally came up with a simple solution: SendGrid would POST to an http endpoint via the API Gateway, which would in turn fire up a Lambda function, which would take the JSON event and write it to an S3 bucket. The reason for S3 instead of something more structured like RDS or SimpleDB is because we can use Splunk to ingest S3 contents. Your requirements may be different, so be sure to check out other storage options like those I have mentioned already.

SendGrid Logging Diagram

The initial plan. The API structure changed, but the flow of events is still accurate.

Now that we have introductions out of the way, let’s jump in and start building this thing. You will need to be familiar with creating Lambda functions and general S3 storage management. Note that I will borrow heavily from the API Gateway Getting Started guide and Lambda with S3 tutorial. Most of my testing took place on my personal AWS account and cost me $.02.

Create an S3 Bucket

The first thing you need to do is create your S3 bucket or folder that will store SendGrid events as files (you can also use an existing bucket). The simple GUI way is to open your AWS console and access the S3 dashboard. From there, click the Create Bucket button. Give your bucket a unique name, choose a region and click Create.

Create a Lambda Function

This won’t be an in-depth guide into creating Lambda functions, but we will cover what you need to know in order to get this up and running. At the time of writing, Lambda supports three languages: Java, Node.js, and Python. I will use Node.js in this guide.

The Code

Create a file called index.js and add the following contents:


//Modified from AWS example: http://docs.aws.amazon.com/lambda/latest/dg/with-s3.html

var AWS = require('aws-sdk');

exports.handler = function(event, context) {
console.log("routeToS3 Lambda function invoked");

//Restrict this function so that not just anyone can invoke it.
var validToken = event.validToken;
//Check supplied token and kill the process if it is incorrect
var token = event.token;
if (token != validToken) {
console.log('routeToS3: The token supplied (' + token + ') is invalid. Aborting.');
context.fail('{ "result" : "fail", "reason" : "Invalid token provided" }');
} else {
uploadBody(event, context);
}
};

uploadBody = function(event, context) {

var bucket = event.bucket;
var app = event.app;
var timestamp = Date.now();
var key = app + '_' + timestamp;
var body = JSON.stringify(event.body);

var s3 = new AWS.S3();
var param = {Bucket: bucket, Key: key, Body: body};
console.log("routeToS3: Uploading body to S3 - " + bucket);
s3.upload(param, function(err, data) {
if (err) {
console.log(err, err.stack);// an error occurred, log to CloudWatch
context.fail('{ "result" : "fail", "reason" : "Unable to upload file to S3" }');
} else {
console.log('routeToS3: Body uploaded to S3 successfully');// successful response
context.succeed('{ "result" : "success" }');
}

});
};

This script will become your Lambda function and has a few key elements to take note of. First, it declares a variable named AWS with “require(‘aws-sdk’)”. This pulls in the aws-sdk Node.js module, which is required for writing to S3. With most Node.js modules, you will need to zip up the module files with your Lambda function. However, the AWS SDK is baked in, so you don’t need to worry about uploading any dependency files with the above function.

Next, the function declares a series of variables, starting with “validToken” and “token.” This might be where most seasoned API engineers roll their eyes at me. When possible, it makes sense to handle authentication at the API level and not inside your function. In fact, the API Gateway has this functionality built in. However, the supported method requires a change to the incoming requests header. That is not an option with SendGrid’s event webhook, which only gives you control over the URL, not the data. So, I had to cheat a little. We will cover this a little more when we setup the API, but for now it is sufficient to understand that token must match validToken for the function to work. Otherwise, the function will exit with an error.

Moving on to the other important variables:

  • bucket – The bucket or bucket/path combination (e.g.: my-bucket/SendGridEvents)
  • app – The name of the app these events are coming from; will be used as the resulting file’s prefix
  • timestamp – The current timestamp, which will be used to make the file name/key unique
  • key – constructed from app and timestamp to generate the file name

All of these variables will be passed in via the API Gateway as part of the event variable. That is why they all look something like “bucket = event.bucket”.

When this script is run, the very first thing Lambda will do is call the “exports.handler” function. In our case, exports.handler simply checks the token and, if it is correct, calls the “uploadBody” function. Otherwise, it exits the script and writes an error to CloudWatch via console.log.

Zip up index.js and use it to create a new Lambda function named “routeToS3.” You can do this all through the GUI, but I am more familiar with the CLI method. Not because I am a CLI snob, but because when Lambda first came out, only account admins could access the Lambda GUI.

Create your API

The AWS API Gateway enables people to build APIs without typing a line of code. It’s really fast to get something up and running. In fact, when all I meant to do was make sure my permissions were set correctly, I accidentally built the whole thing. I recommend checking out AWS’s guide, but you can also learn a bit by following along here.

To start…

  1. Log into your AWS console and open up the API Gateway service and click the Create API button.
  2. Name your API routeToS3 and click the next Create API button.
  3. With the root resource selected (it should be your only resource at this point), click Actions -> Create Resource.
  4. Name the resource “input” and set the path to “input” as well.
  5. Select /input from Resources menu on the left.
  6. Click Actions -> Create Method.
  7. In the dropdown that appears on the Resources menu, select POST and click the checkmark that appears to the right.
  8. For Integration Type, choose Lambda Function.
  9. Set your Lambda Region (choose the same region as your S3 bucket).
  10. Type or select the name of your Lambda function (routeToS3) in the Lambda Function field.
  11. Click Save
  12. When prompted to Add Permission to Lambda Function, click OK.

Congratulations! You just built an API in about two minutes. Now, in order to make sure the Lambda function gets all the parameters we mentioned earlier (body, bucket, app, etc.), we need to configure query strings, a mapping template, and a stage variable. We won’t be able to create a stage variable just yet, so that will come a little later.

With your POST method selected in the Resources menu, you should see a diagram with boxes titled Method Request, Integration Request, Method Response, and Integration Response:

POST Function

Click on Method Request to setup our query strings. From here, click to expand the URL Query String Parameters section. Any query string we add here will act as what some of us might refer to as GET parameters (e.g.: /?var1=a&var2=b&var3=etc). To setup the strings we will need, follow these steps:

  1. Click the Add query string link.
  2. Name the string token and click the checkmark to the right.
  3. Repeat for app and bucket.

Go back to the method execution overview by clicking POST in the Resources menu or <- Method Execution at the top.

Next, we will add a mapping template:

  1. Click Integration Request.
  2. Expand the Body Mapping Templates section.
  3. Click Add mapping template
  4. Type application/json (even though it is already filled in and doesn’t disappear when you click inside the text box) and click the checkmark to the right.
  5. Click the pencil icon next to Input Passthrough (it’s possible you could see “Mapping template” instead).
  6. Add the following JSON object and click the checkmark

{
"bucket": "$input.params('bucket')",
"app": "$input.params('app')",
"token": "$input.params('token')",
"validToken": "$stageVariables.validToken",
"body": $input.json('$')
}

This mapping will take the body of the request and our variables, and pass them along as part of the event object to Lambda. Note that all values, like “$input.params(‘bucket’)” are wrapped in double quotes, except for $input.json(‘$’). That is because we are actually calling a function on the body (‘$’), so wrapping it in quotes will break things.

Now, it’s time to deploy our API, which will make it accessible over HTTP. But, we haven’t tested it yet and that validToken variable is still undefined. Don’t worry, we haven’t forgotten those two critical pieces. But, we have to create a stage first, which is part of the deployment process.

  1. Click the Deploy API button at the top of the screen.
  2. On the screen that appears, choose [New Stage] for the Deployment Stage.
  3. Choose a name for the stage (Stages are like different environments, for example dev or prod).
  4. Enter a Deployment description and click Deploy.

On the screen that follows, you will see a tab labeled Stage Variables. Open this tab and click Add Stage Variable. Name the variable validToken and enter a token of your choosing for the Value. Use something strong.

Go back to the Settings tab and take a look at the options there. You may be interested in throttling your API, especially if this is a development stage. Remember that, although the API Gateway and Lambda are fairly cheap, too much traffic could rack up a bill. Since we aren’t using a client certificate to authenticate the calling app, we have to invoke the Lambda function to verify the provided token. Just something to keep in mind when considering throttling your API.

Now that I’ve distracted you with some prose, click Save Settings at the bottom of the page.

At the top of the screen, you will see an Invoke URL. This is the address to access the stage you just deployed into. All of our magic happens in the /input resource, so whatever that invoke URL is add “/input” to the end of it. For example, https://yudfhjky.execute-api.region.amazonaws.com/dev would become https://yudfhjky.execute-api.region.amazonaws.com/dev/input.

With our stage setup, we can now test the method.

  1. Go back to the routeToS3 API and click on the POST method in the Resources menu.
  2. Click Test.
  3. Enter a token, app, and a valid bucket/folder path (e.g.: my-bucket/routeToS3/SendGrid)
  4. Enter a value for validToken (this should be the same as token if you want the test to succeed).
  5. For Request Body, type something like {“message”: “success”}.
  6. Click Test.

You should see the CloudWatch logs that indicate the results of your test. If all is well, you will get a 200 status back and a corresponding file will appear the bucket you provided. The file contents should be {“message”: “success”} or whatever you set for the request body.

If things are working as expected, then it is time to head over to SendGrid and configure your event webhook:

  1. Log into SendGrid.
  2. Click Settings -> Mail Settings.
  3. Find Event Notification.
  4. Click the gray Off button to turn event notifications on.
  5. If needed, click edit to enter the HTTP POST URL.
  6. Enter the URL to your API endpoint, along with all necessary query strings (e.g.: https://yudfhjky.execute-api.region.amazonaws.com/dev/input?bucket=my-bucket/routeToS3/SendGrid&token=1234567890&app=SendGrid).
  7. Click the checkmark.
  8. Check all the events you want to log.
  9. Click the Test Your Integration button.
  10. Wait a couple minutes and then check your bucket to see if SendGrid’s test events arrived.

Tada! You should now be logging SendGrid events to an S3 bucket. Honestly, it’s much simpler than you might think based on the length of this post. Just keep the perspective that all of this is accomplished with three lightweight and low-cost services: the API Gateway to receive the event from SendGrid, Lambda to process that event and upload it to S3, and S3 to store the body of the SendGrid event. I hope you find this as helpful and straightforward as I have.

Gumper – an API for executing scripts remotely

Two things I enjoy in technology: PowerShell and web development. I have written before about bringing these two things together by using PowerShell to generate json data. Gumper, as I will explain, makes it easier for sysadmins and web developers to take advantage of this technique by providing an API to run scripts on-demand and get json as output.

Gumper GUI

Over the past couple years, I have been working on an internal support tool that launches PowerShell scripts through a web-based GUI. While development has continued on this project, it became clear that its ability to run PowerShell scripts needed to be separated from the application itself (MVC, right?). Gumper is the solution to that need. Gumper, licensed under the MIT license, maintains a registry of local scripts (currently Bash and PowerShell) on a server, which can be run using their unique ID (generated by Gumper) and an auth key provided via GET. Gumper will also accept arguments for the script.

Why is this important for sysadmins and web/front-end developers? I have seen a lot of sysadmins write great scripts and web developers create really useful tools. But, very rarely have I seen a web developer have both the expertise and access to integrate a sysadmin’s script output into a handy tool. Gumper makes this possible with minimal work on either end: sysadmins already know how to write scripts and web developers already know how to work with an API and parse json. And, it just makes the whole process easier. Help Desk needs to see which user accounts are disabled? Easy. Write a script and throw it into Gumper. CFO wants to know how many pages a department is printing? Gumper! Well… at least to get the results. Then, something else should actually parse it into an understandable format (coming soon…).

The important thing to know is: It’s easy to make your existing scripts work with Gumper. You can read more on the project wiki, but if your scripts are written in PowerShell, it’s insanely simple. All you need to do is add the following output to your script (keep in mind, this doesn’t work with interactive scripts that use a lot of read-hosts; that should be built into an interface):


...

$json = $object | convertto-json#Where $object represent the results you want to return

Write-Host "[[OUTPUT]]"

$json

Write-Host "[[/OUTPUT]]"

Gumper will return only the output between the two OUTPUT tags. Now, add your script to Gumper’s registry (you cannot upload scripts via Gumper, so they must already exist on the server) and you can get the results using something like this:


https://gumper.domain.org/?script=559ae260a6005192965742389&authKey=DefaultChangeMePlease!args=smith;john

Your response might look something like this:


{"authentication":"success","script":"559ae260a6005192965742389","scriptResult":[{"lastName":"Smith","firstName":"John","department":"Finance","userName":"jsmith"}]}

Note the authKey used in the above example. That is the default and you should change it before making Gumper available! To change it, just open keys.psrconfig and change…


"value":"DefaultChangeMePlease!"

… to something unique.

So, those are the basics, but you should really checkout the README and wiki on Bitbucket if you are interested in using Gumper. It is still very new, so please allow for some glitches and feel free to submit feedback, fork the project, etc. It may not be exciting on its own, but paired with some front-end development (hopefully, my next project to be released), it has some important implications.

jQuery getJSON from PowerShell via PHP on IIS: A Frustrating Gotcha

(Once you have your PHP on IIS environment setup and ready to go, you can check out this example code. You can also take a look at my new project, Gumper, on Bitbucket – an API for executing PowerShell and other scripts remotely.)

And in 40 years, this subject line will make absolutely no sense whatsoever…

About two years ago, I wrote a PowerShell script that generated a .Net form with all sorts of tools for our Help Desk. By searching for an Active Directory user, the Help Desk could instantly see if the user’s account or password was expired, whether or not the user had a mailbox, if the user’s mailbox database was mounted, quota information, and more. Recently, I revisited the script and decided it was time to take it to the web. Running these tasks from a PowerShell GUI is not terribly flexible or efficient. So, I set out to learn what it seems many Sys Admins like myself are learning: how to run PowerShell scripts via PHP.

Right out of the gate, I am already committing a bit of heresy. Rather than installing WAMP, I decided to stick with the IIS instance where I had already installed PHP (http://blogs.iis.net/bills/archive/2006/09/19/How-to-install-PHP-on-IIS7-_2800_RC1_2900_.aspx). And why use PHP to run Microsoft PowerShell (built on .NET)? Quite frankly, I like PHP and find it easy to learn. This project is still in its infancy, but there have already been a few important snags I thought worth sharing.

Use shell_exec()

If you want to launch a PowerShell script from a PHP file, you have a few options: exec(), shell_exec(), and system(). Using system() doesn’t seem to make sense if you intend to get a response from the server. This would be intended more for something like kicking off a scheduled task or another background process. Exec() will do the job, but it will split your response up into an array based on line breaks. This might be OK depending on how you want data returned. But, for my purposes, I chose shell_exec() so I could format the data a bit. Shell_exec() will return the output of your script as a string.

Keep your script silent

Note that “shell_exec()” returns all output from your script. That means errors, write-hosts, and everything else that pops up when you run a script. So, be sure to make your script as silent as possible and only “return” the little bit of data you want passed into PHP. This might mean a lot of Try/Catches (which is a good practice anyway).

Launch your script

This was actually a pretty easy one to conquer. Many people have examples of the basic syntax to use in order to launch a PowerShell script. Here is an example of what I am using:

shell_exec(“powershell -NoProfile -File $scriptPath $argString < NUL”)

(You may also need to pass the “-ExecutionPolicy” switch with a value depending on your setup.)

The most critical part of this is the “< NUL” bit at the end. Without it, PHP will never get the output from PowerShell and will, instead, wait and wait and wait. You will also notice that I use two variables: $scriptPath and $argString. These are PHP variables that I pass into a function and are used to call the script file along with any arguments. So, if $scriptPath is “C:\web\script1.ps1” and $argString is “-User jdoe”, the above line would render as:

shell_exec(“powershell -NoProfile -File C:\web\script1.ps1 -User jdoe < NUL”)

Authenticate

Remember that IIS has different options for authentication. I chose Basic with HTTPS, but someone else may have a better idea since I really just went with whatever worked first. The main thing is to turn off Anonymous authentication. The reason Basic works well for my situation is that each page runs as the authenticated user and their permissions. The importance of this will become evident below.

Enable fastcgi.impersonate

Even if you are authenticated, you probably won’t be able to launch any PowerShell scripts by default. This is because PHP does not pass along your authentication to the Windows command line unless fastcgi.impersonate is enabled. Enabling this in php.ini makes it so every PHP script that runs, runs as the authenticated user. Keep that in mind, because it may change how you design your site.

To enable fastcgi.impersonate, locate your PHP.ini file (Maybe C:\Program Files (x86)\PHP\v5.x\php.ini) and un-comment the line that says “;fastcgi.impersonate = 1;” by removing the semicolon (;) at the beginning of the line. The line should look like this when you are done:

fastcgi.impersonate = 1;

After this, save php.ini and restart your web site.

Return JSON

If you are going to manipulate the data inside the browser, it makes sense to return your data from PowerShell as JSON. This is pretty straightforward. For example:


$a = "apple"

$b = "banana"

$c = "coconut"

$json = "{`"a`": `"$a`",`"b`": `"$b`",`"c`": `"$c`"}"

Return $json

This should spit out the following:


{"a": "apple", "b": "banana", "c": "coconut"}

Be sure to wrap string in escaped quotations (`”) to keep your JSON valid. And validate your syntax with JSONLint (http://jsonlint.com/).

Beware the Console Width

This is the real reason for writing; the thing that nearly made me go brain-dead. I chose to use jQuery’s getJSON function since I was already returning a JSON array. There are many great tutorials that show you how to accomplish this, so I won’t get into that. Despite all the great tutorials, I was getting nowhere. No matter what I did–callback to a function, change the mode to synchronous (WHAT!?), use $.ajax, click with my right pinky-finger–nothing worked. I could see the data in FireBug, but I could not get anything to show up on the page. This frustrated me all night. Today, I was watching the same data pass through FireBug, refreshing, clicking again, and feeling utterly hopeless (OK, so maybe there are more important things in life than PowerShell and PHP), when I finally realized something important: maybe FireBug wasn’t wrapping that really long Distinguished Name in the JSON array for easy reading. Maybe the data was coming back to the browser with a real line break, thereby invalidating the JSON.

Yup, that was it.

It turns out, when shell_exec() launches PowerShell, the data returned is formatted to the default console size: 80 characters wide. Meaning, if your JSON object goes beyond 80 characters, it will break to the next line of the console and the data you get back will be invalid. For example:


{"distinguishedName": "OU=weeny,OU=teeny,OU=bitsy,OU=itsy,DC=reallylongnamiccusd

omainiccus,DC=net"}

Instead of…


{"distinguishedName": "OU=weeny,OU=teeny,OU=bitsy,OU=itsy,DC=reallylongnamiccusdomainiccus,DC=net"}

So, what are the options? Well, there are two I can think of. First, build well-formed line breaks into your JSON like so:


$a = "apple"

$b = "banana"

$c = "coconut"

$json = "{`n`"a`": `"$a`",`n`"b`": `"$b`",`n`"c`": `"$c`"`n}"

Return $json

Adding the new line character (`n) will cause the JSON to break in a spot that will not invalidate the array. This should return:


{

"a": "apple",

"b": "banana",

"c": "coconut"

}

Option 2 is to adjust the console size if your script will output more than 80 characters on a line. The “Hey, Scripting Guy!” blog has a great article on how to do this: http://blogs.technet.com/b/heyscriptingguy/archive/2006/12/04/how-can-i-expand-the-width-of-the-windows-powershell-console.aspx

As I said earlier, this project is in its infancy and I am sure many more gotchas await me. However, this last one seemed like something others might run into. It’s so silly, but so frustrating. And, I imagine anything run from the command prompt will yield the same results.

Happy coding.

listMedia: an HTML5/JavaScript Function

I attended a brief workshop in Grand Rapids yesterday with Phil Gerbyshak. He talked about sharing one idea with someone everyday, which is why I feel motivated to share this bit of JavaScript with the Interwebs.

As I was going over some HTML5 tutorials, I became excited about the video tag. True, it does not add anything that could not be accomplished with HTML4, but it does it in a much cleaner way. Perhaps my favorite part is the built-in intelligence with recognizing supported media types. Say you aren’t sure whether an OGG or MP4 is better to use for compatibility sake. HTML5 says, “why not use them both?” By using the following code, the visitor’s browser will automatically select the very first file type it recognizes:

<video width="320" height="240" controls="controls">
  <source src="movie1.mp4" type="video/mp4" />
  <source src="movie1.ogg" type="video/ogg" />
</video>

If the browser does not recognize movie1.mp4, it will simply move onto movie1.ogg. And for the record, “controls=’controls'” and “source src=” have to be some of the most redundant HTML tags I have seen in my life.

This is all great, but it got me thinking. Even though there are only three primary video formats (kinda) supported in HTML5 (OGG, MP4, and WebM), what if this list grows? Five years form now, we don’t want to write out fifteen lines of possible video sources “just in case.” So, I set out to create a JavaScript that would do this for us. I call it listMedia and it currently has a strict and loose version.

listMedia Strict

This code would go into your <head> or an external JS file:

<script type="text/JavaScript">
function listMedia(mediaAddr, mediaType)
{
if (mediaType="video")
   {
   var vidFormats= new Array("ogg","mp4","webm");
   for(var i=0;i<vidFormats.length;i++)
      {
      document.write("<source src='" + mediaAddr + "." + vidFormats[i]
      + "' type='video/" + vidFormats[i] + "' />");
      }
   }
else if (mediaType="audio") //checks to see if the media type is audio
   {
   var audFormats= new Array("ogg","mp3","wav");
   for(var i=0;i<audFormats.length;i++)
      {
      document.write("<source src='" + mediaAddr + "."
      + audFormats[i] + "' type='audio/" + audFormats[i] + "' />");
      }
   }
else //media type failed to be detected
   {
   document.write("<source src=
   'http://myweb.arbor.edu/jthiede/no-type.mp4' />");
   }
}
</script>

Then, you would place something like this in your <body>:

<video width="320" height="240" controls="controls">
   <script type="text/JavaScript">
      listMedia('http://www.w3schools.com/html5/movie', 'video');
   </script>
Your browser does not support the video tag.
</video>

What’s happening?

Starting form the bottom and going up… A bit of JavaScript nested in the <video> tag calls the “listMedia” function and passes it two variables: most of a URL (http://www.w3schools.com/html5/movie) and the type of media (video). These variables get processed by listMedia and checked to see if the media is a video or audio file. Since the mediaType is set as “video,” the variables go through the first IF condition.

An array called vidFormats is used to keep up with all the currently supported video formats. Just a note – this array can be added to and stripped down to your heart’s delight without messing anything up. Finally, listMedia prints out a list throwing the appropriate extensions on the end of the mediaAddr variable. Voila! No need to type out every format.

listMedia Loose (aka Version 3)

You may notice that there is a perfectly useless “else” condition in the strict version. What was supposed to happen was that a video saying “please ask the web admin to select the proper media type” would show up if the media type was not selected or was invalid. However, while debugging this, I learned that it was not required to specify whether the media was audio or video, so long as your <audio> and <video> tags were used correctly. I was amazed to find that HTML5 allows a browser to check a media file against any extension, video or audio. Meaning, you can keep one array with all the media extensions, and your visitor’s browser will find the right one.

Thus, I created a simplified version of listMedia. My purpose for retaining both versions is that I do not know how strict HTML5 will become and I grow weary of bright red boxes on validators for not having something like an <alt> tag. For now, the loose version works great. However, this may change:

<script type="text/JavaScript">
function listMedia(mediaAddr)
{
   var mediaFormats= new Array("ogg","mp3","mp4","wav","webm");
   for(var i=0;i<mediaFormats.length;i++)
      {
      document.write("<source src='" + mediaAddr + "." + mediaFormats[i] + "' />");
      }
}
</script>

<audio controls="controls">
   <script type="text/JavaScript">
      listMedia('http://www.w3schools.com/html5/song');
   </script>
Your browser does not support the audio element.
</audio>

Smaller and more efficient. This is basically the same process, but we no longer care if the media is audio or video. Instead, we use one array to define ALL of the media extensions. The key for getting both versions to work is remembering not to pass the full path of your file to the mediaAddr variable. Instead of typing “http://mysite.com/video1.ogg&#8221; just type “http://mysite.com/video1&#8221;. listMedia will take care of the rest.

That’s really it for now. Honestly, so long as their are only five supported media formats, this may not be a huge time saver. But, it could make a big difference in the future. My next two small-scale projects will be using listMedia with CSS3 and porting listMedia over to PHP (which seem much more applicable IMHO).

Please feel free to offer any suggestions. To see listMedia in action, go to the terribly unflattering http://myweb.arbor.edu/jthiede/listMedia.htm. View the source of the page to see all the comments that I could not get to fit in WordPress.

Enjoy!