File storage and video hosting on Amazon S3

Posted on by

Amazon S3 (Simple Storage Service) provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. Each object is stored in a bucket and retrieved via a unique, developer-assigned key. It’s possible to write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited.

The pricing plan for Amazon S3 is based on “Pay only for what you use”. There is no minimum fee. The prices vary across geographic regions and are based on the location of the bucket. For Europe the pricing effective november 2008 is as follows :

Storage :

  • $0.180 per GB – first 50 TB / month of storage used
  • $0.170 per GB – next 50 TB / month of storage used
  • $0.160 per GB – next 400 TB / month of storage used
  • $0.150 per GB – storage used / month over 500 TB

Data Transfer :

  • $0.100 per GB – all data transfer in
  • $0.170 per GB – first 10 TB / month data transfer out
  • $0.130 per GB – next 40 TB / month data transfer out
  • $0.110 per GB – next 100 TB / month data transfer out
  • $0.100 per GB – data transfer out / month over 150 TB

Requests :

  • $0.012 per 1,000 PUT, POST, or LIST requests
  • $0.012 per 10,000 GET and all other requests

The Amazon S3 service is part of the Amazon Web Services, an infrastructure web services platform in the cloud. The direct link to the S3 login page is aws.amazon.com/s3/. I created my account on Amazon S3 on october 13th, 2008.

Some test files to verify the correct configuration are accessible at the following links :

The sender of a request to an AWS web service is authorised by Access Identifiers. Access identifiers are also used to authenticate requests to AWS. For services that require authenticated requests, where you need to prove that you are authorized to make the request, you must sign the request by including a “signature” in the request. A request signature is calculated using the pair of public / private Access Identifiers. AWS supports two types of Request Identifiers: AWS Access Key Identifiers or X.509 Certificates.

The Access Key ID is used as the value of the AWSAccessKeyId parameter in requests you send to Amazon Web Services and identifies you as the party responsible for the request. Since the Access Key ID is not encrypted in requests to AWS, it could be discovered and used by anyone. Services that are not free require you to provide additional information, a request signature, to verify that a request containing your unique Access Key ID could only have come from you. You use your Secret Access Key to calculate a signature to include in requests to web services that require authenticated requests.

Uploading and managing files on the Amazon S3 distributed system is not straight forward. Amazon offers documentation, release notes, sample code & libraries, developer tools, articles & tutorials and a community forum as resources to developers.

A comfortable user interface to upload and manage files on the 3S servers called “Bucket Explorer” (version 2008.06.30.01) is offered by Chambal.com.Inc for Windows, Linux and Mac (website www.bucketexplorer.com). The price for one licence is USD 49.95, a 30 day free trial version is available. A free plugin for the Firefox browser called “S3Fox Organizer” (version 0.4.1) has been developped by Rjonna of Suchi Software Solutions from India.

Another Firefox plugin (Gspace) developped by the same author and distributed by Fon Labs allows you to use your Gmail Space (2700 MB) as a remote machine. You can transfer files between your hard drive and gmail. Your gmail account looks like a FTP host and you can upload and download your files. After you install, you get an option called “GSpace” in your “tools” menu clicking on which opens the window for transfer. You have to be aware that this extension is not endorsed by Google and that this usage is not compliant with the Google Terms of Service.

BlackBerry RSS reader

Posted on by

Viigo Inc. develops and markets one of the world’s most popular mobile content and services applications for smartphones. With over 5,000 channels and information services, Viigo customers around the world enjoy up‐to‐the minute access to news, weather, sports, stocks and markets, entertainment, and blogs in one intuitive, simple, freely-downloadable application.

Viigo offers a free beta version of an RSS reader for BlackBerry. This third party application is also used by RIM executives, as stated by crackberry.com, the #1 site for BlackBerry users & abusers.

Crop or padd a video

Posted on by

Last update : August 2013, 2013

Sometimes it’s necessary to crop or padd a video to change the aspect ratio and the resolution. The size of a video image is measured in pixels for digital video, or horizontal scan lines and vertical lines of resolution for analog video. In the digital domain standard-definition television resolution is specified as 720/704/640 ×480 for NTSC and 768/720 × 576 for PAL or SECAM. However in the analog domain, the number of visible scanlines remains constant (486 NTSC/576 PAL) while the horizontal measurement varies with the quality of the signal: approximately 320 pixels per scanline for VCR quality, 400 pixels for TV broadcasts, and 720 pixels for DVD sources. Aspect ratio is preserved because of non-square pixels.

High-definition televisions are capable of resolutions up to 1920 ×1080 pixels.

Aspect ratio describes the dimensions of video screens and video picture elements. All popular video formats are rectilinear, and are described by a ratio between width and height. The screen aspect ratio of a traditional television screen is 4:3. High definition televisions use an aspect ratio of 16:9.

Some common used video resolutions and aspect ratios are :

  • 128 : 96 > 4 : 3
  • 176 : 144 > 11 : 9
  • 240 : 176 > 4 : 3
  • 320 : 240 > 4 : 3
  • 352 : 288 > 11 : 9
  • 480 : 272 > 16 : 9
  • 640 : 480 > 4 : 3
  • 720 : 480 > 3 : 2

To keep the aspect ratio of a video without distortion when changing the resolution, you often need to crop or to padd the video (delete or add pixels on top, bottom, left and right side). I use the great video-tool Super from eRightSoft to do this job. The current version is v2013.build.57 released on July 13, 2013.

DVD Video media

Posted on by

Last update : November 4, 2013

DVD Properties

DVD

DVD

DVD Video is a consumer video format used to store digital video on DVD (DVD-ROM) discs, and is currently the dominant form of consumer video formats in the world.

A DVD contains 3 type of files: VOB, IFO and BUP.
A VOB file (Video Object) is a container format contained in DVD-Video media. It’s the core file of a DVD and contains the actual Video, Audio, Subtitle, and Menu contents in stream form. VOB is based on MPEG-2 program stream format, however, VOB files are a very strict subset of the standard. An overview to show the logical structure of VOB files is given on the MPUcoder website.

An IFO file (InFOrmation) stores information about Chapters, Subtitles and Audio Tracks. It contains important navigational information, such as where a video chapter begins, and where audio and subtitle streams exist within the movie (VOB) file. IFO files are not encrypted.

A BUP file (Back UP) is a backup of the IFO file and is used in the event that the corresponding IFO file is unreadable, perhaps due to a scratch on the surface of the disc.

A DVD contains the following data files :

  • VIDEO_TS.IFO : The ‘Video [Title Set] Manager Information Set’ is a configuration file that defines the disc structure and also includes region coding information and aspect ratio etc
  • VIDEO_TS.VOB : The ‘Video [Title Set] Object file for VMG Menu’, this file works like a bookmark to tell the player that it is the start of the disc.
  • VIDEO_TS.BUP : Backup file of the’ Video Manager Information Set’ (VIDEO_TS.IF0).
  • VTS_01_0.IFO : The’ Video Title Set Information Configuration` file for the first lot of VOB files, it also tells the player how to proceed when using the on screen menu system
  • VTS_01_0.VOB : ‘Video Object Set for Video Title Set Menu’, the first video and audio stream sequence to be played usually the on screen menu system. These files may contain several streams of audio/video “multiplexed” (MUX) together (eg. chapters/language selections)
  • VTS_01_0.BUP : Backup file of the’ Video Title Set Information’ file (VTS 01_1.IFO)
  • VTS_01_1.VOB : The first ‘Video Title Object Set’ file, the main file containing the picture and audio streams. It is usually split into three to four files, each with ascending filenames; for example VTS_01_1.VOB, VTS_01_2.VOB, VTS_01_3.V0B and so on. No VOB file is to exceed 1GB in size, with a maximum total of five VOB files in sequence on a single layer disc (DVD 5) and no more than nine on a dual layer disc (DVD 9). VOB files have a naming standard VTS_xx_yy.VOB (xx = Title number, yy = Section number).

A DVD can also include additional Video Title Sets that include the IFO, VOB and BUP with ascending filenames such as VTS_02_1, VTS_03_1 and so on. These are usually the extra features that come with the title, such as documentaries and behind-the-scenes footage. Each title has at least 2 VOBs (one for the menu and one for the main feature) and exactly one IFO and BUP file.

An freeware editor IfoEdit (version 0.971) for IFO-files is available at www.ifoedit.com. A complementary program PgcEdit (version 9.3) with more possibilities, called the swiss knive of DVD edition, is available at the videohelp.com website. A guide how to create IFO files from VOB files with IfoEdit is available at the digital-digest website.

A very detailed description (in german) about DVD’s is available at the dvd-tipps-tricks website.

DVD Resolution

Whether a Display Aspect Ratio (DAR) of 4:3 or 16:9, the native resolution of a DVD is 720 x 576 pixels (5:4) for PAL/SECAM (Europe) or 720 x 480 pixels (3:2) for NTSC (US). The DAR is registrated in the IFO-file and handled by the television or the DVD player to present the video in the correct aspect ratio.

For PAL, a 4:3 video ist stretched to 768 x 576 pixels, a 16:9 video is stretched  to 1.024 x 576 pixels. For NTSC, a 4:3 video is resized to 640 x 480 pixels, a 16:9 video is stretched to 854 pixels.This process is called anamorphic widescreen.

If the DVD shows Panavision (ratio 2.20:1), CinemaScope (2.39:1) or other widescreen films (1.66:1, 1.85:1, 2:35:1, 14:9, …), hard-matting or black bars adding techniques (letterboxing) are used to include the image within the 16:9 format.

DVD authoring

DVD authoring is the process of creating a DVD video that can be played on a DVD player. DVD authoring software must conform to the specifications set by the DVD Forum group in 1995. The specifications are complicated due to the number of companies that were involved in creating them.

There are a lot of DVD authoring, encoding and burning programs available: professional, commercial, proprietary, free of charge and open source software. Among the free programs, I prefer  DVD Flick (only for Windows available), developped by Dennis Meuwissen (pseudo : Exl). The latest version is 1.3.0.7.

DVD Forum

The DVD Forum is an international organization composed of hardware, software, media and content companies that use and develop the DVD formats. It was initially known as the DVD Consortium when it was founded in 1995.

DVD Formats

The following main DVD formats are available :

  • DVD-R : recordable ; capacity 4,7 GB; approved by the DVD-Forum
  • DVD-RW : rewritable ; capacity 4,7 GB; approved by the DVD-Forum
  • DVD+R : recordable ; capacity 4,7 GB; developed by the DVD+RW Alliance
  • DVD+RW : rewritable ; capacity 4,7 GB; developed by the DVD+RW Alliance

There are a number of significant technical differences between the dash and the plus format, although most DVD drives can handle both formats and most users would not notice the difference. The write time for a full disc depends on the maximal speed supported by the media; the current maximum speed (Sony DVD) is 24x (31.68 MB/s).

Flash Video (FLV) : delivery and encoding

Posted on by

Last update : Januray 30, 2013

Due to the popularity of the Adobe Flash player, Flash video (FLV) was the standard of choice for Internet Video publishers in the past. Publishers could choose between three methods of delivering video content.

  • Progressive Download
  • Streaming
  • HTTP pseudo-streaming

Downloads are easy and work with any server, but do not offer content protection nor seeking to undownloaded parts. Streaming servers offer these functionalities, but have to be installed and managed. Since serverside access and knowledge are not available to every developer, there exist an in-between solution of HTTP pseudo-streaming through the use of small serverside scripts. A detailed tutorial about HTTP video streaming has been written by Jeroen Wijering, the renowned developer of the JW player and cofounder of LongTail Video and Bits on the Run.

The streaming protocol for Flash video is RMTP (Real Time Messaging Protocol). RTMP is a proprietary persistant protocol developed by Adobe Systems for streaming audio, video and data over the Internet, between a Flash player and a server.

The RTMP protocol has three variations:

1. The “plain” protocol which works on top of TCP and uses port number 1935
2. RTMPT which is encapsulated within HTTP requests to traverse firewalls
3. RTMPS which works just like RTMPT, but over a secure HTTPS connection.

The HTTP pseudo-streaming works through a very simple mechanism. It’s a progressive download using server side scripting to allow random access to parts of the video that have not yet been loaded. A well known php script for HTTP pseudo-streaming was xmoov, developed by Eric Lorenzo Benjamin jr. The FLV video must contain keyframes metadata which can be added with a patcher like FLVMDI.

The second issue for delivering flash video on the web is the encoding. An in-depth comparison of available codecs is presented by Jan Ozer in an article published on the website digitalcontentproducer.com.

Flash video uses three different codecs:

  • The Sorenson Spark codec, introduced in Flash version 7, (Sorenson H.263) was used a lot (e.g. at YouTube, Google Video, MySpace). This codec was also found in most freeware / shareware software that encodes FLV.
  • The TrueMotion VP6 codec developed by On2 Technologies was introduced in Flash version 8. It was a famous proprietary codec used in the Flix encoding tools offered by On2. Two versions of the VP6 were available: VP6-S was recommended for HD (high-definition) production, and VP6-E was the best for SD (standard definition) production.
  • The industry-standard H.264 video codec is icluded in Flash since version 9r115. This codec is also known as MPEG-4 Part 10, or MPEG-4 AVC (for Advanced Video Coding). With H.264, DVD-quality online video is very well possible. An advantage of the H.264 codec is that it is also used in a lot of portable devices, such as the iPod/iPhone, Sony PlayStation Portable (PSP) or Nokia N series smartphones.

A comparison of the quality of H.263 and 264 codecs is presented by Jeroen Wijering on the Bits on the Run website.

I use the commercial version of Jeroen Wijerings video-player (licence No 2007.3264). My video files are hosted on the Amazon S3 servers, the preview pictures are hosted in the same folder as the video player on another server by using a relative path for the corresponding flashvariable. Since 2010 I use the Amazon Cloudfront service to stream flash videos.

TWITTER : what are you doing ?

Posted on by

Twitter is a service for friends, family, and co–workers to communicate and stay connected through the exchange of quick, frequent answers to one simple question: What are you doing?
You can use 140 characters or less to send your thoughts, observations, and goings-on in your day. Whether you’re “eating an apple” or “looking foward to the weekend” or “Heading out of town”, it’s twitter-worthy. Twitter messages are called tweets.

Hashtags are a community-driven convention for adding additional context and metadata to the tweets. An online hashtag is created by prefixing a word with a hash symbol: #hashtag.

Followers (followers_me) are those who have elected to receive tweets from a twitter, following_me are friends whose tweets a twitter is following.

Twitter can be used on the web or with a mobile phone (SMS), it’s interworking with Facebook and it’s supporting an API to develop specific applications. Twitter offers an search and advanced search webpage to find out what’s happening in the world beyond your personal timeline. To search, filter, and otherwise interact with the volumes of news and information being transmitted to Twitter every second, Twitter Search helps to filter all the real-time information coursing through their service.

Twitter is a privately funded startup with offices in the SoMA neighborhood of San Francisco, CA. Started as a side project in March of 2006, Twitter has grown into a real-time short messaging service that works over multiple networks and devices.

Detailed informations are available on Wikipedia.

Layers over objects using z-index

Posted on by

last update : june 13, 2011

I noticed that a layer with an embedded webpage as an object is not covered by an other layer having a higher z-index in Internet Explorer, whereas it is correctly displayed in Firefox. The same problem has been reported in the webmasterworld forum. I found a workaround (see www.artgallery.lu/expositions/expo_samuel_martin.html) to integrate the text from the second layer in the embedded webpage, but this works only without links.

Another tough nut to crack for developers is the z-index working with Flash, because the Flash Player always writes the SWF on the topmost layer of the page, regardless of z-index order. It works with swfobject if the parameter wmode is set to opaque (see an example on the website of Leslie’s Artgallery).

A demopage showing how z-index works is available at the tjkdesign.com website.

Email Encoder and Spam Blocker for Web Forms

Posted on by

Spam is a great problem for all website owners. A simple method to stop email harvesters from picking up e-mail addresses from web pages consist in encoding the email address in hexadecimal characters. By encoding the entire e-mail address including the “mailto” portion, most e-mail harvesters (except the most sophisticated) can’t find it. If they can’t find it, they can’t send spam to it.

A free hexadecimal email encoder is offered on the website from K’nechtology Inc. 

To protect web-forms from spam abuse, more sophisticated solutions are required. Spam Blocker For Web Forms is  a powerful, easy-to-use application ideal for protection of web forms from being parsed and used for sending spam emails. Includes adjustable spam filters and anti spam source code protection. I use a registered version since more than a year with great success.

Spam Blocker For Web Forms is developped by Dmitry Konovalov from White Stranger Group. The current version is 1.1, a single user license is available for 40 US$, a business license for 60 US$.

Oplayer : mobile media player

Posted on by

Last update : January 30, 2013

In 2001, I tested the first version of the Oplayo Mobile Video Player on mobile phones with Symbian operating system. My first test phone was the Nokia 7650. The videos were converted from standard video files into Oplayo’s MVQ format using the Oplayo Media Designer. The videos typically required a mobile data connection with 26 kbps bandwith and used the HTTP protocol for streaming.

Oplayo has been at the forefront of mobile media development and became a global leader in non-standard encoded mobile video technology. In 2010 the Oplayo domains were acquired by MediaWireless.