DICOM Testing with DVTk

Last update : December 11, 2015

DVTk history

DVTk is an Open Source project for testing, validating and diagnosing communication protocols and scenarios in medical environments. It supports DICOM and IHE integration profiles. The project started in 1997 by Philips and was joined by Agfa in 2001. In 2005, DVTk was open sourced and one year later ICT Automatisering joined the open source community. in 2009 Agfa left the project.

The following tools are currently free available from the DVTK.org website or from SourceForge (Windows binaries and library sources) :

  • DICOM Compare, v 3.0.3
  • DICOM Editor, v 3.2.6
  • DICOM Network Analyzer, v 3.3.2
  • DVT Validator, v 2.6.8
  • Query Retrieve SCP Emulator, v 4.1.3
  • RIS Emulator, v 3.2.0
  • Storage SCP Emulator, v 3.3.0
  • Storage SCU Emulator, v 3.3.0

DVTk Tests

I used these tools in the context of the development of an education tool for radiologists called RadioLogic. I edited the following script to execute an ECHO test and a STORE-SCP test with the Orthanc DICOM Server, version 0.9.5 :

# DVT Dicom Script
SESSION
SESSION-TYPE emulator
SESSION-FILE-VERSION 1
# Product Test Session Properties
SESSION-TITLE "Orthanc Server"
SESSION-ID 001
MANUFACTURER "Manufacturer"
MODEL-NAME "Product Name"
SOFTWARE-VERSIONS "Version 1.0"
APPLICATION-ENTITY-NAME "ORTHANC"
APPLICATION-ENTITY-VERSION "0.9.5"
TESTED-BY "Marco Barnig"
DATE "20151208"
# SUT ACSE Properties
SUT-ROLE acceptor
SUT-AE-TITLE "ORTHANC"
SUT-MAXIMUM-LENGTH-RECEIVED 16384
SUT-IMPLEMENTATION-CLASS-UID ""
SUT-IMPLEMENTATION-VERSION-NAME ""
# DVT ACSE Properties
DVT-AE-TITLE "DVTK_STR_SCU"
DVT-MAXIMUM-LENGTH-RECEIVED 16384
DVT-IMPLEMENTATION-CLASS-UID "1.2.826.0.1.3680043.2.1545.1"
DVT-IMPLEMENTATION-VERSION-NAME "DVT2.6"
# Socket Properties
SUT-HOSTNAME "localhost"
SUT-PORT 4242
DVT-PORT 104
DVT-SOCKET-TIMEOUT 30
USE-SECURE-SOCKETS false
TLS-VERSION "TLSv1"
CHECK-REMOTE-CERTIFICATE true
CIPHER-LIST "aRSA+kRSA+SHA1+eNULL:@STRENGTH:-SSLv2"
CACHE-TLS-SESSIONS true
TLS-CACHE-TIMEOUT 300
CREDENTIALS-FILENAME ""
CERTIFICATE-FILENAME ""
# Test Session Properties
LOG-ERROR true
LOG-WARNING true
LOG-INFO true
LOG-RELATION false
LOG-DEBUG false
LOG-DULP-STATE false
LOG-SCP-THREAD true
PDU-DUMP false
STORAGE-MODE as-media
STRICT-VALIDATION false
DETAILED-VALIDATION-RESULTS true
SUMMARY-VALIDATION-RESULTS true
INCLUDE-TYPE-3-NOTPRESENT-INRESULTS false
AUTO-TYPE-2-ATTRIBUTES true
DEFINE-SQ-LENGTH false
ADD-GROUP-LENGTH false
# Supported Transfer Syntaxes
SUPPORTED-TRANSFER-SYNTAX "1.2.840.10008.1.2"
SUPPORTED-TRANSFER-SYNTAX "1.2.840.10008.1.2.2"
SUPPORTED-TRANSFER-SYNTAX "1.2.840.10008.1.2.1"
# Configurable Delay between N-Action and N-Event Command
DELAY 10
# Definitions
DEFINITION-DIRECTORY "%COMMONPROGRAMFILES%\DVTk\Definition Files\DICOM\"
DEFINITION "All DIMSE Commands.def"
DEFINITION "CT Image Storage.def"
DEFINITION "Digital X-Ray Image Storage - For Presentation.def"
DEFINITION "Digital X-Ray Image Storage - For Processing.def"
DEFINITION "Enhanced CT Image Storage.def"
DEFINITION "Enhanced MR Image Storage.def"
DEFINITION "Media Storage Directory.def"
DEFINITION "MR Image Storage.def"
# Results
RESULTS-ROOT ".\results\"
APPEND-TO-RESULTS-FILE false
# DICOMScript Description Directory
DESCRIPTION-DIRECTORY ".\html\"
ENDSESSION

Scripts are saved as text files with the extension .ses. Several scripts can be referenced in an xml-project file with the extension .pdvt.

The following figure shows the resulting settings in the DVTk session panel :

Session

DTVk DICOM validator session panel

By double-clicking on the Storage-SCU-emulator link at the left, a window pops up allowing to do an ECHO test or to send a DICOM file to the Orthanc server. The exchanged messages are displayed in the Activity Logging panel :

DVT

DVTk Activity Logging

After the succesful sending of a DICOM file to the Orthanc server, a summary of the validation results is shown in the Validation Results panel :

DVT

DVTk Validation Results

In verbose mode, the Orthanc Server provides the following releated messages in the command window :

Orthanc Server output in verbose mode

Orthanc Server output in verbose mode

DVTk Network Sniffer

The next figure shows the results of the DVTk network sniffer analyzing the DICOM traffic between the OSIRIX HD viewer on iPad with the ORTHANC DICOM server (version 0.9.6) running on Window 8.1.

DVTk

DVTk summary results of FIND Request sent by Osirix HD from iPad

I entered only the first characters of the patient name to send a FIND query. DVTk reports four errors in the C-FIND communication because the following data elements are not valid :

  • (0008, 0018) : SOP Instance UID > should be present with at least a zero length
  • (0010, 0020) : Patient ID > should be present with at least one value
  • (0020, 000D) : Study Instance UID > should be present with at least one value
  • (0020, 000E) : Series Instance UID > should be present with at least one value

The C-MOVE communication works without errors and the received results displayed in Osirix HD are correct.

DVTk

DVTk reports no errors in the C-MOVE communication between Osirix HD and Orthanc

The Orthanc server shows no specific error message in the command window.

The OSIRIX HD viewer is configured as follows :

  • Remote AET : ORTHANC
  • IP : 192.168.178.26
  • Port : 4242
  • AETitle : OSIRIXHD
  • IP : 192.168.178.65
  • Port : 4096
  • Transfer Syntax : Implicit Little Endian (preferred syntax by Orthanc)
  • Encoding : ISO IR 100 (Latin-1)
  • Protocol : C-MOVE
  • Timeout : 90 sec

Communication Failure

The next example shows the case of a failed communication. The C-FIND query sent by the PACS DICOM iPad app to the Orthanc server (version 0.9.6) is aborted with a DIMSE failure. The query is identical to the C-FIND request sent by Osirix HD. The Orthanc log signals “DIMSE No data available (timeout in none-blocking mode)”.

PACS DICOM

Orthanc log messages related to a C-FIND command sent by the PACS DICOM app

The DVTk network analyzer shows the same four errors as in the case of the Osirix communication. After a timeout, the communication is aborted.

DVTk

DVTk signals an Abort Request in the PACS DICOM – Orthanc association

The PACS DICOM viewer is configured as follows :

  • Server AET : ORTHANC
  • Host Name/IP : 192.168.178.26
  • Server Port : 4242
  • Local AET : TEST-SCU
  • Device IP Address : 192.168.178.65
  • Local Port : 4096
  • Protocol : C-MOVE

A deeper analysis is required to identify the reason of the abort request. If the problem is caused by a non-valid parameter in the C-FIND query, the recently proposed LUA script filter will be an excellent remedy.

Accessing Local Virtual Hosts

Last update : September 7, 2015

Local Networks

A local area network (LAN) is a computer network that interconnects computers within a limited area such as a residence, school, laboratory, or office building. It is contrasted to a wide area network (WAN), which covers a larger geographic distance and may involve leased telecommunication circuits. Ethernet over twisted pair cabling and Wi-Fi (WLAN) are the two most common transmission technologies in use for local area networks.

Virtual Hosts

The term Virtual Host refers to the practice of running more than one website (such as dev.radiologic.fr and local.radiologic.fr) on a single machine. The fact that they are running on the same physical server is not apparent to the end user. Apache was one of the first servers to support virtual hosts right out of the box.

Computer Name, Address and Route

To find and access a computer in an network, we must know the following informations :

  1. Name : to indicate what we seek ( > Domain Name )
  2. Address : to indicate where it is ( > IP Address )
  3. Route : to indicate how to get there ( > Routing )

Domain Name

Domain names are used to identify one or more IP addresses. They are used in URLs to identify particular web pages. Domain names can be local or public. The latter must be registered with an ICANN accredited registrar.

A domain name consists of one or more parts, technically called labels, that are conventionally concatenated, and delimited by dots, such as example.com. The right-most label conveys the top-level domain (TLD), including the generic top-level domains (gTLDs), such as the prominent domains com, info, net, edu, and org, and the country code top-level domains (ccTLDs) such as lu, de, fr.

Below the top-level domains in the domain name hierarchy are the second-level domain (SLD) names. These are the names directly to the left of the top-level domains. Second-level domain names are usually created based on the name of a company, product or service. There can be additional sub-level domains with virtually no limitation. Below these levels, the most right domain name component is used to designate a particular host server, for example www for a world wide web server, ftp for an FTP server, or any other label. A domain name with TLD, SLD and hostname is called a fully qualified domain name (FQDN), for example dev.radiologic.fr.

Local domain names used in local networks can be freely chosen by the network administrator, but some TLD’s should be avoided. For example Apple uses the domain extension .local for it’s implementation of zero-configuration networking called Bonjour.

To map domain names to IP addresses, a Domain Name System (DNS) is used.

IP Address

An IP Address (Internet Protocol address) is a numerical label assigned to each device participating in a computer network that uses the Internet Protocol for communication. The designers of the Internet Protocol defined an IP address as a 32-bit number (IPv4) which is still in use today. A new version of IP (IPv6), using 128 bits for the address was developed in 1995 to avoid a depletion of the available addresses. Its deployment has been ongoing since the mid-2000s.

IP addresses are usually written and displayed in human-readable notations, such as 172.16.254.1 (IPv4), and 2001:db8:0:1234:0:567:8:1 (IPv6).

An IP address is logically recognized as consisting of two parts: the network prefix and the host identifier, or interface identifier (IPv6). The routing prefix is expressed in CIDR (Classless Inter-Domain Routing) notation. It is written as the first address of a network, followed by a slash character (/), and ending with the bit-length of the prefix. For example, 192.168.1.0/24 is the IPV4 prefix starting at the given address, having 24 bits allocated for the network prefix, and the remaining 8 bits reserved for host addressing.

IPv4 reserves some addresses for special purposes such as private (local) networks or multicast addresses. The following ranges are reserved for private networks :

Prefix length Start End
/8 10.0.0.0 10.255.255.255
/12 172.16.0.0 162.31.255.255
/16 192.168.0.0 192.168.255.255

In IPV6, the reserved address blocks for private networks are referred to as unique local addresses (ULA) and they use the routing prefix fc00::/7.

IP addresses can be static (fix) or dynamic. Static IP addresses are manually assigned to a computer by the network administrator. Dynamic IP addresses are assigned either by a server (router) using Dynamic Host Configuration Protocol (DHCP) or by the computer itself, as in Zeroconf. In the last case of address autoconfiguration (link-local address) the address block 169.254.0.0/16 is reserved in IPv4 networks and the block fe80::/10 in IPV6 networks.

Subnets

In IPV4 a subnet mask is used to indicate how the IP address is divided into network and host parts. A subnet mask is made by setting network bits to all “1”s and setting host bits to all “0”s. For example, 255.255.255.0 is the network mask for the 192.168.1.0/24 prefix.

Routing

Routing is the process of selecting best paths in a network. Routing schemes differ in their delivery semantics:

  • unicast : destination is a single specific node in the network
  • broadcast : destinations are all nodes in the network
  • multicast : destinations are all interested nodes in the network
  • anycast : one to many routing topology
  • geocast : destinations are all nodes in a geographical area

Unicast is the dominant form of message delivery on the Internet.

DNS Resolver

The main job of a DNS server is to store DNS name data and serve it when it receives requests. The main job of a DNS resolver is to transform a DNS name into an IP address, but this is only one of several types of resolution services performed by DNS. The process to resolve an hostname to an ip address is normally called dns lookup.

Reverse DNS lookup or reverse DNS resolution (rDNS) is the determination of a domain name that is associated with a given IP address.

Accessing local virtual hosts

There are different ways that local websites can be viewed from other computers which are connected within the same LAN. These computers can be desktops or laptops running Windows, Mac OSX, Linux or be mobile devices like tablets (iPad, Samsung, …) or smart phones (iPhone, Android, …).

  • Use the LAN IP of the server : downside – only one website can be accessed
  • Use the LAN host name of the server : downside – only one website can be accessed
  • Use a LAN-wide hosts file : downside – this doesn’t work on iOS and most other mobile devices
  • Use own DNS server : downside – setup is complex ; DNS server must always be turned on
  • Use a Wi-Fi router with DNS Masquerading : downside – only a few routers support this option
  • Use registered domain names : downside – all website domain names must have a common base; works only with Internet access
  • Use a HTTP proxy server : downside – multiple

We are analyzing now what solution to adapt for a real network case, related to the specific RadioLogic project.

RadioLogic project requirements

RadioLogic is an university education tool for radiologists which uses three websites, working as virtual hosts :

  • www.radiologic.fr : hosted on Amazon AWS, this website provides the tools and data for public users
  • local.radiologic.fr : hosted on a local server, this website provides the tools and data for private users on a local network
  • dev.radiologic.fr : hosted on a local server, this website provides the beta versions of new tools and data for testing before deployment on the public and private hosts

If possible, the IT consumerization policy BYOD (Bring your own device) is promoted allowing the radiologists in residence to use their own devices. The local server is a MacBookAir laptop running OSX Yosemite 10.10.5 or later. The clients are iPads running iOS 8.1.4 or later. Both the server and the clients will be used for other purposes.

There exist four operational modes :

  1. The server is used as standalone client without Internet connection
  2. The server is used as standalone client with an Internet connection through a Wi-Fi router
  3. An iPad is used as client with an Internet connection through the same Wi-Fi router (same local network)
  4. An iPad is used as client with a local AdHoc Wi-Fi Connection (SSID = radiologic-open) established by the server

The following figure shows the four operational modes.

Four

RadioLogic : four operational modes

The project requirement is that the same host names are used in each operation mode to access the three websites. Let’s examine how to achieve this goal.

Server without Internet connection used as a client

Without Internet connection, it’s not possible to access the public website www.radiologic.fr. To access the local websites local.radiologic.fr and dev.radiologic.fr, the most simple solution is to use the internal hosts file available on every desktop computer (Windows, Linux, Mac OSX, …). This configuration file for the loopback interface is located at /private/etc/hosts on the Mac OSX, but it’s also accessible through the more traditional location /etc/hosts. This file is configured as follows :

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	localhost
127.0.0.1       local.radiologic.fr
127.0.0.1       dev.radiologic.fr
255.255.255.255	broadcasthost
::1             localhost

The default values are the IP addresses for the localhost in IPV4 and IPV6 and the broadcasthost for IPV4. The radiologic hosts have been added.

Server with Internet connection used as a client

Accessing the public website www.radiologic.fr is now possible and the preceding loopback solution is also working.

iPad used as a client with Internet connection in local network

There is no problem to access the public website www.radiologic.fr, but the simple loopback solution to access the local hosts is not possible on mobile devices without jailbreaking.

The next possibility consists in using a local DNS server. Setting up a DNS server on the Mac laptop with primary zones for the local hosts needs a parameter change in the DNS configuration of the iPad Wi-Fi connection.

Wi-Fi

Wi-Fi settings in iPad

We must change the IP address of the DNS server, assigned by the Internet provider via the Wi-Fi router, by the local IP address of the Mac laptop, which leads to various problems :

  • The IP address of the Mac laptop changes when you use it in a different local network or when the DHCP server assigns a new address in the same network at booting
  • The DNS server of the Mac laptop must be continuously running to use the iPad for Internet surfing
  • Using an additional DNS server with forwarding to a public DNS server to set up an Internet connection may lead to delays or performance problems
  • Frequent changes of the DNS addresses on an iPad is an error-prone process for a normal user

I recommend to never change the network parameters assigned by the Internet provider on your router and local devices to avoid all sort of problems.

There exist a third solution to access the local hosts in our case, which some developers consider as a hack : using the public DNS server to point towards the internal private IP address of the Mac computer.

This solution is simple and feasible in our case because all hosts use the same domain name radiologic.fr which is registered and managed in a public DNS server. Instead of creating an A record for both dev.radiologic.fr and local.radiologic.fr we can use a wild-card (*) to catch all DNS records.

Some reasons to not use a public DNS server with local IP addresses, invoked by other developers, are the following :

  • leak of internal informations to third parties
  • it’s not considered best practice
  • external clients may inadvertently be redirected to a host on their own local network
  • it’s likely to cause confusion and human miscommunication
  • risk of mail delivery timeouts or mail bounces
  • risk of DNS rebinding attacks

I don’t agree with these arguments that I consider as minor problems or not applicable in specific cases. I think the solution is not a such bad idea in practice and I will use it in the RadioLogic project.

Some routers provide a security feature to protect against DNS rebinding attacks by suppressing DNS responses that refer to IP addresses in the home network. My router FRITZ!Box 7390 is such a device. It allows to enter domain names in a list of exceptions where this protection should be disabled. In our case these are dev.radiologic.fr and local.radiologic.fr.

fri

Exceptions to the DNS rebinding protection in the Fritz!Box router 7390

iPad used as a client connected to an AdHoc network

The only workable solution in this case is the use of a local private DNS server. You may ask what about the problems listed in the preceding chapter ? Well, these are no longer issues in the case of an AdHoc Wi-Fi network :

  • The self-assigned IP address of the AdHoc Wi-Fi interface in the range 169.254.0.0/16 can be defined as static
  • The DNS server is always running when the Mac laptop runs the AdHoc network
  • Without Internet connection there is no forwarding to a public DNS server
  • The DNS address is changed once in the configuration of the AdHoc Wi-Fi connection which is different from the configuration of the other Wi-Fi infrastructure connections

Search Domain

In iOS and OSX Wi-fi settings, each Wi-fi connection has a Search Domains section which is simply a convenience that allows the system to convert host names to Fully Qualified Domain Names (FQDN). Anyplace you might use a FQDN, such as in the location bar of Safari,  you can use a simple host name. If the system can’t resolve that host name to an IP address, then it will try appending the search domain to the host name and try again. If you specified more than one search domain, it will try them each in order.

If we set the search domain to radiologic.fr, we can use the host names dev and local to access the corresponding websites. I didn’t opt for this solution because it introduces more complexity and an additional delay in setting up connections.

Network Diagnose Tools

There are several tools available for iPads to diagnose access problems in local networks:

net

iPad Network Analyzer : ping statistics – LAN overview

Links

The following list provides links to websites with additional informations about accessing local virtual hosts :

Mac OSX Wireless Networks

Last update : September 17, 2015

AdHoc Wi-Fi Networks

Sometimes it’s useful to create a WLAN (wireless network) between two or more Wi-Fi-enabled computers without using an Wi-Fi router. These computer-to-computer networks are usually referred to as AdHoc wireless networks.

create

Create an ad-hoc Wi-Fi network on Mac OSX

On a Mac OSX (Yosemite 10.10.5) it’s easy to set up such a network. Choose Create Network from the Wi-Fi status icon in the menu bar. Enter a name (in my case : radiologic-open) for the new network and select the channel (default channel = 11). You will notice that there is no password protection to secure the network. A WEP (Wired Equivalent Encryption) algorithm was however available in previous versions of the OSX system. WEP was Wi-Fi’s first encryption standard and was almost intentionally designed to be weak due to issues related to the US export policies about encryption. WEP was deprecated in 2003 and replaced by WPA2 (Wi-Fi Protected Access) by the Wi-Fi Alliance.

In Yosemite, Apple killed WEP once for all, without replacing it by another protection method in AdHoc mode.

open

Chose name and channel for the AdHoc network

If the icon isn’t in the menu bar, choose Apple menu > System Preferences, then click Network (réseau). Click Wi-Fi and select the Show Wi-Fi status in menu bar checkbox.

Mac OSX

Mac OSX System Preferences

The Wi-Fi status menu shows the established connection with the radiologic-open network. The menu allows an easy logout and reconnection to another Wi-Fi network. An automatic logout occurs when the Mac screen is powered off.

Radoio

Wi-Fi status menu shows the AdHoc network active

Infrastructure Wi-Fi networks

There exist a second possibility to set up a Wi-Fi network on Mac OSX. The computer is configured as a software Wi-Fi base station. This allows to share an Internet connection and offers a password protection, but it has another inconvenience. You need a working wired Ethernet connection between your Mac and a hub, even if the hub is not connected to the Internet. The next picture shows my test setup: my MacBookAir is connected via a Ethernet-Thunderbolt interface to an ancient 10 Mbit/s Ethernet hub.

Thunderbolt-Ethernet

MacBookAir connected with Thunderbolt-Ethernet interface to hub

First you need to configure the Ethernet-Thunderbold interface. Go to Network in System Preferences and select the automatic configuration. The interface gets a self-assigned IP address in the range 169.254.x.x.

network

Ethernet-Thunderbolt automatic configuration

In the next step go to the Sharing Preference Pane in the System Preferences. Select Ethernet-Thunderbold as source port and Wi-Fi as destination port. Select Wi-Fi options to enter a name (in my case : radiologic-secure) of the network, to select a channel and to define a password (minimum 8 characters; numbers are not recognized on BlackBerry and Android).

Internet Sharing Wi-Fi network configuration

Internet Sharing Wi-Fi network configuration

Finally check the Internet sharing checkbox and confirm the settings to activate the connection.

Share

Activate the Internet sharing

The resulting pane is shown below.

rads

Activated Internet sharing pane

The Wi-Fi status is updated automatically. To log out, desactivate the Wi-Fi connection in the corresponding menu. An automatic logout occurs when the Mac screen is powered off.

Radio

Wi-Fi status menu showing Internet sharing

The following table shows which of my devices are capable to connect to the Mac OSX AdHoc and Infrastructure networks.

Device Wi-Fi AdHoc Wi-Fi infrastructure
iPad OK OK
iPhone OK OK
BlackBerry  network not shown network joined, but
no access to host
Samsung Tablet (Android 4.2.2) network not in range network joined, but
no access to host
Laptop Vista no connection network joined, but
no access to host
Desktop Windows 8.1 network not shown OK
Laptop Debian no connection OK

Wi-Fi Diagnostic

To detect the reasons why some devices don’t connect to the Mac OSX Wi-Fi AdHoc or Infrastructure networks, you can use a WLAN analyser. Mac OS X Yosemite has an in-built Wi-Fi scanner to help you find the best Wi-Fi channel. When you hold down the option key ⌥ (next to the CTRL key) and select the Wi-Fi icon in the menu bar, a secret dropdown menu opens.

secret

Secret Wi-Fi dropdown menu in Mac OSX Yosemite

Open Wireless Diagnostics and click it. After you have gotten to the page and have opened up the Wireless Diagnostics window, go to the top left of your menu bar and click on Window where you find several options: Informations, history, scan, performance, detection, …  Some results are shown below.

Mac OSX Wi-Fi Analyser

Mac OSX Wi-Fi Analyser

Another WLAN tool is available on Windows computers. When you enter the command

netsh wlan show networks mode=bssid

in the command window, you will get the following results :

Wi-Fi

Microsoft Wi-Fi Analyser

A third WLAN tool is integrated in my Wi-Fi router FritzBox 7390. Various wireless networks in my neighborhood are shown, but nor the channels used by the Mac OSX nor the names of these networks are listed. I don’t know why ?

funkkanale

Wi-Fi channels used in proximity of the Fritzbox

There are also several external software tools for Wi-Fi-analysis available, for example Acrylic Wi-Fi Free or Acrylic Wi-Fi Professional. The main window (Access points,  Signal strength) of Acrylic is shown hereafter :

Acrlylic Wi-Fi Analyser

Acrlylic Wi-Fi Analyser

Some additional windows (stations, packet viewer, 2,4 GHz APs channels, detailed info) are shown below :

Various Acrylic results

Various Acrylic results

Wi-Fi Assessment

Several methodologies are available dealing with safety aspects of wireless networks, for example :

Links

The following list shows links to websites providing additional informations about Wi-Fi networks, related to Mac OSX.

Volunteer Computing

Volunteer computing is an arrangement in which people (volunteers) provide computing resources to projects which use the resources to do distributed computing and/or storage. Distributed computing is a field of computer science that studies distributed systems. A distributed system is a software system in which components, located on networked computers, communicate and coordinate their actions by passing messages.

Neural networks are very good candidates for simulation by distributed computing systems because of their inherent parallelism and beacuse its simulation is a very time consuming process, due to the complex iterative process involved.

The first volunteer computing project was the Great Internet Mersenne Prime Search, which was started in January 1996. The term volunteer computing was coined by Luis F. G. Sarmenta, the developer of Bayanihan.

The Berkeley Open Infrastructure for Network Computing (BOINC) is the most widely-used middleware system for volunteer computing. It offers client software for Windows, Mac OS X, Linux, and other Unix variants. The project was founded at the University of California, Berkeley Space Sciences Laboratory, funded by the National Science Foundation. Other systems are XtremWebXgrid and Grid MP.

Volunteer computing systems must deal with the following problems, related to correctness :

  • Volunteers are unaccountable and essentially anonymous
  • Some volunteer computers occasionally malfunction and return incorrect results
  • Some volunteers intentionally return incorrect results or claim excessive credit for results

A list of distributed computing projects is provided at Wikipedia. Links to a few selected BOINC volunteer computing projects are listed below :

oEmbed

oEmbed is a format for allowing an embedded representation of a URL on third party sites. The simple API allows a website to display embedded content (such as photos or videos) when a user posts a link to that resource, without having to parse the resource directly.

An oEmbed exchange occurs between a consumer and a provider. A consumer wishes to show an embedded representation of a third party resource on their own web site, such as a photo or an embedded video. A provider implements the oEmbed API to allow consumers to fetch that representation. The response returned by the provider can be in either JSON or XML.

The following types are defined :

  • photo
  • video
  • link
  • rich

The authors of oEmbed are Cal Henderson, Mike Malone, Leah Culver and Richard Crowley. Among the providers of oEmbed are Youtube, Flickr, Vimeo, WordPress and Slideshare.

UPnP Technology and Test Tools

Universal Plug and Play (UPnP) is a set of networking protocols that permits networked devices, such as personal computers, printers, gateways, servers and mobile devices to seamlessly discover each other’s presence on the network and establish functional network services for data sharing, communications, and entertainment. UPnP is intended primarily for residential networks. No special device driver support is necessary, common network protocols are used (TCP/IP, HTTP, XML, SOAP). UPnP uses UDP port 1900 and TCP port 2869.

The UPnP technology is promoted by the UPnP Forum. Formed in October 1999, the UPnP Forum is an industry initiative of more than 1000 leading companies in computing, printing, networking, consumer electronics and home appliances. A huge library of UPnP resources is available at the website of the UPnP Forum.

UPnP was published as an international standard, ISO/IEC 29341, in December 2008.

The foundation for UPnP networking is IP addressing, either by DHCP or by AutoIP (zero configuration networking). The next step in UPnP networking is discovery, based on the Simple Service Discovery Protocol (SSDP). SSDP allows a device to advertise its services to control points, for example to media servers (Serviio) on the network. The next step is to learn more about the discovered device and its capabilities by retrieving the device’s XML description from the URL provided by the device in the discovery message. Now the control point can send actions to a device’s service using the Simple Object Access Protocol (SOAP).

An additional capability of UPnP networking is event notification, or eventing. The event notification protocol defined in the UPnP Device Architecture is known as General Event Notification Architecture (GENA). The final step in UPnP networking is presentation, usually by providing an URL of a webpage loaded in a browser.

UPnP AV is an audio and video extension of UPnP, referenced in specifications published by other organizations, including the Digital Living Network Alliance (DLNA).

Problems with UPnP are lack of authentication and vulnerabilities with the risk that UPnP devices can be accessed and/or manipulated by malicious web sites.

A complete set of open source tools for creating and testing UPnP solutions is provided by Intel. They include Device Spy, a generic control point, and Device Builder, a code generation tool. In all, there are 11 tools in this package that range from low-level debugging to full reference tools :

  • Device Spy
  • Device Sniffer
  • Network Light
  • Device Builder
  • Device Relay
  • Device Validator
  • Service Author
  • AV Media Controller
  • AV Media Server
  • AV Media Renderer
  • AV Wizard

Network Light is a software model of an UPnP device to show the basic functions of the UPnP protocol. The light can be switched on/off or dimmed. The following snaps show how

network_light

UPnP Network Light OFF / ON / DIMMED

UPnP Device Spy Tool

UPnP Device Spy Tool

setLoadLevelTarget

UPnP Device Spy Tool : Invoke action DIM 50%

UPnP Device Spy Tool : Invoke action ON

UPnP Device Spy Tool : Invoke action ON

UPnP Device Sniffer Tool

UPnP Device Sniffer Tool

Another UPnP test tool, called UPnP Inspector, has been developed by Coherence. The Inspector is an UPnP Device and Service analyzer, and a debugging tool, based on the Coherence DLNA/UPnP framework. It is loosely modeled after the Intel UPnP Device Spy and Device Validator.

UPnP Inspector

UPnP Inspector

UPnP Inspector logs

UPnP Inspector logs

UPnP Inspector action

UPnP Inspector action

UPnP Inspector details

UPnP Inspector details

 

DAS, SAN, RAID and NAS

Last update : September 11, 2013

A Direct-attached storage (DAS) is a file-level computer data storage system directly attached to a server or workstation, without a storage network in between. The main protocols used for DAS connections are ATA, SATA, eSATA, SCSI, SAS, and Fibre Channel.

A storage area network (SAN) is a dedicated network that provides access to consolidated, block level data storage. A SAN does not provide file abstraction, only block-level operations.

A Redundant array of independent disks (RAID) is a storage technology that combines multiple disk drive components into a logical unit. Data is distributed across the drives in one of several ways called RAID levels, depending on what level of redundancy and what performance is required. In RAID 1, data is written identically to two drives, thereby producing a mirrored set.

A Network-attached storage (NAS) is a file-level computer data storage system connected to a computer network, providing data access to a heterogeneous group of clients. A NAS, in contrast to a SAN, uses file-based network sharing protocols such as NFS, SMB, CIFS or AFP. NAS systems are networked appliances which contain one or more hard drives, often arranged into logical, redundant storage containers or RAID arrays. The benefits of network-attached storage, compared to file servers, include faster data access, easier administration, and simple configuration.

The storage space can be comprised of USB or hard disk drives. A NAS can be sold with the drives included or without them. Like PCs, NAS units have memory and processors. With better processors and increased memory, you get better performance from a NAS, just as with PCs. Lunix is usually used as embedded operating system. Noise and security are two other important concerns.

In the last years, NAS devices have been gaining popularity in the home office (SOHO), because they are useful for more than just general centralized storage provided to client computers in environments with large amounts of data. Typical other usages are :

  • load balancing
  • email server
  • web server
  • media server
  • multimedia streaming
  • cloud-based backup
  • low-cost video surveillance
  • BitTorrent client

A list of NAS manufacturers is provided at Wikipedia. A list of links to additional informations about SAN’s is provided below :

DLNA Media Servers and Players

Last update : March 12, 2013

The following list provides links to informations about DLNA media devices, especially Media Servers :

A list of additional media servers and client applications is available on Wikipedia which provides also a comparisons of UPnP AV media servers.

Apple developed its own proprietary streaming protocol stack/suite, called AirPlay, that is not compliant with DLNA. Another solution to stream multimedia files is based on the SMB/CIFS networking protocols, re-implementated with Samba to open windows to a wider world.

I recommend the Serviio DLNA server which I installed a week ago on my two Windows 7 desktops to replace Windows Media Players.

Windows Media Player DLNA Configuration

Last update : June 17, 2012
A personal computer running Windows XP, Windows Vista or Windows 7 can be used as a DLNA certified media server simply by activating this capability in the Windows Media Player (WMP) 11 or above. Some tutorials how to configure the WMP are listed hereafter :

In the past the media server components, now integrated in the WMP, were called Windows Media Connect (WMV) or Home Media Ecosystem (HME). The Windows Media Center (WMC) includes a digital video recorder.

Late April 2012 I set up a DLNA system in my home network with a Fritz!Box Media Server and two Windows Media Players 12 running on Windows 7 personal computers.

Six weeks later I replaced the Windows Media Player system by the outstanding Serviio DLNA server. I disabled the Windows 7 Media Player Network Sharing Service (wmpnetwk) to avoid conflicts in media sharing and to speed up the system’s perfomance. A tutorial how to turn off wmpnetwk.exe is available on numerous websites.

DLNA : Digital Living Network Alliance

Last update : June 17, 2012;

DLNA logo

The Digital Living Network Alliance (DLNA) allows consumers to connect and enjoy their music, videos and photos from any consumer device (computers, printers, cameras, cell phones, and other multimedia devices) from anywhere in their homes. DLNA (website : dlna.org) is a non-profit collaborative trade organization established by Sony in June 2003, that is responsible for defining interoperability guidelines to enable sharing of digital media between these devices . The private guidelines are built upon existing public standards and specify a set of restricted ways of using the standards in order to achieve interoperability.

Today there are 26 promoter members and 199 contributor members. In early 2011, DLNA began a Software Certification program in order to make it easier for consumers to share their digital media across a broader range of products. Today there are over nine thousand products on the market that are DLNA certified.

DLNA uses Universal Plug and Play (UPnP) for media management, discovery and control.

The DLNA Certified Device Classes are separated as follows :

  • Digital Media Servers (DMS) store content and make it available to networked media devices
  • Digital Media Controller (DMC) find content on digital media servers and play it on digital media renderers
  • Digital media renderers (DMR) play content received from a digital media controller
  • Digital media players (DMP) find content on the network and provide playback and rendering capabilities
  • Digital media printers (DMPr) provide printing services to the DLNA home network

Examples for DMS include personal computers or network-attached storage (NAS) devices. All DLNA devices can be mobile (wireless). Bridges between mobile handheld device network connectivity and home network connectivity are provided by  Mobile Network Connectivity Functions (M-NCF). Content transformations between required media formats for home network and mobile handheld devices iare provided by Media Interoperability Units (MIU).

DLNA is a refinement of UPnP, a set of rules and restrictions in the name of interoperability. Full DLNA specifications are available only to DLNA members who pay for. UPnP specifies the abstract device interfaces, the specifications for UPnP are available at the UPnP forum.