...
        LOG(LOG_DEBUG, "Internal Redirect starting: uri (%s)", filepath);
        apr_table_set(r->subprocess_env, "IS_PROCESSED", "true");
        ap_internal_redirect(filepath, r);
        LOG(LOG_DEBUG, "Internal Redirect finished: uri (%s)", filepath);
...

The apr_table_set function call sets an environment variable named 'IS _PROCESSED' to the value 'true'. Seems simple, right? I had some mod_rewrite rules that looked for an environment variable named 'IS_PROCESSED' to direct the handling of the request:

RewriteCond %{ENV:IS_PROCESSED} true
RewriteRule ....

The mod_rewrite documentation states that environment variables can be accessed using an ENV prefix:

"%{ENV:variable}, where variable can be any environment variable, is also available. This is looked-up via internal Apache structures and (if not found there) via getenv() from the Apache server process."

Unfortunately, there are some details that are omitted from the documentation; the RewriteCond directive above will not work. When calling the ap_internal_redirect function, all of the environment variables currently associated with the request are renamed (see http_request.c) to use the prefix "REDIRECT_". I imagine this is intended to distinguish the pre-redirect and post-redirect environment variables. It would be nice if this were documented some place.

So, the mod_rewrite directives ended up looking like:

RewriteCond %{ENV:REDIRECT_IS_PROCESSED} true
RewriteRule ...

This is another example of how the Apache HTTPD Server is incredibly flexible, but surprisingly lacking in the way of documentation and consistency.]]> http://urlgrey.net/archives/2010/06/evaulating_apac.php http://urlgrey.net/archives/2010/06/evaulating_apac.php Computing Wed, 16 Jun 2010 13:46:29 -0800 I've made a demonstration video showing how easy it is to configure program subscriptions for MythPodcaster. The Web-based configuration is very straightfoward, and the generated RSS feeds containing links to the transcoded videos can be viewed in any RSS reader. The feeds display very nicely in Apple's Mobile Safari RSS reader. The RSS feeds from MythPodcaster can bridge MythTV with any RSS compatible client, such as the Boxee media player. I might add a Boxee demo at a later date.

In the meantime, here's a demo showing iPad and iPod Touch playback of ATSC over-the-air broadcasts that were recorded with MythTV and transcoded using MythPodcaster.

There are some quirks to the open-source segmenter. First, it doesn't support the segmenting of audio-only input files (i.e. an AAC or MP3 input file). Second, the durations of the transport-stream segments that are reported in the M3U8 playlist are often incorrect. The small duration deviations accumulate when applied to a clip that is 30 minutes or more in total duration.

I've made some improvements to the open-source segmenter to address the issues stated earlier. The complete patched source file can be accessed here. Many thanks go to Chase Douglas (original author of the segmenter) and Carson McDonald (made the segmenter popular).

The unified diff of the segmenter source file is shown below:


--- segmenter.c 2010-05-18 09:51:57.932132172 -0700
+++ segmenter-accurate_durations.c 2010-05-18 09:50:19.765611032 -0700
@@ -20,6 +20,8 @@

#include "libavformat/avformat.h"

+#define MAX_TS_FILES_DEFAULT 1000
+
static AVStream *add_output_stream(AVFormatContext *output_format_context, AVStream *input_stream) {
AVCodecContext *input_codec_context;
AVCodecContext *output_codec_context;
@@ -79,7 +81,7 @@
return output_stream;
}

-int write_index_file(const char index[], const char tmp_index[], const unsigned int segment_duration, const char output_prefix[], const char http_prefix[], const unsigned int first_segment, const unsigned int last_segment, const int end, const int window) {
+int write_index_file(const char index[], const char tmp_index[], const unsigned int segment_duration, const char output_prefix[], const char http_prefix[], const unsigned int first_segment, const unsigned int last_segment, const int end, const int window, const unsigned int actual_segment_durations[]) {
FILE *index_fp;
char *write_buf;
unsigned int i;
@@ -111,7 +113,7 @@
}

for (i = first_segment; i <= last_segment; i++) {
- snprintf(write_buf, 1024, "#EXTINF:%u,\n%s%s-%u.ts\n", segment_duration, http_prefix, output_prefix, i);
+ snprintf(write_buf, 1024, "#EXTINF:%u,\n%s%s-%u.ts\n", actual_segment_durations[i-1], http_prefix, output_prefix, i);
if (fwrite(write_buf, strlen(write_buf), 1, index_fp) != 1) {
fprintf(stderr, "Could not write to m3u8 index file, will not continue writing to index file\n");
free(write_buf);
@@ -194,6 +196,8 @@
fprintf(stderr, "Maximum number of ts files (%s) invalid\n", argv[6]);
exit(1);
}
+ } else {
+ max_tsfiles = MAX_TS_FILES_DEFAULT;
}

remove_filename = malloc(sizeof(char) * (strlen(output_prefix) + 15));
@@ -280,13 +284,15 @@

dump_format(oc, 0, output_prefix, 1);

- codec = avcodec_find_decoder(video_st->codec->codec_id);
- if (!codec) {
+ if (video_index >=0) {
+ codec = avcodec_find_decoder(video_st->codec->codec_id);
+ if (!codec) {
fprintf(stderr, "Could not find video decoder, key frames will not be honored\n");
- }
+ }

- if (avcodec_open(video_st->codec, codec) < 0) {
+ if (avcodec_open(video_st->codec, codec) < 0) {
fprintf(stderr, "Could not open video decoder, key frames will not be honored\n");
+ }
}

snprintf(output_filename, strlen(output_prefix) + 15, "%s-%u.ts", output_prefix, output_index++);
@@ -300,9 +306,11 @@
exit(1);
}

- write_index = !write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, last_segment, 0, max_tsfiles);
+ unsigned int *actual_segment_durations = malloc(sizeof(unsigned int) * MAX_TS_FILES_DEFAULT);
+ write_index = !write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, last_segment, 0, max_tsfiles, actual_segment_durations);

do {
+ unsigned int current_segment_duration;
double segment_time;
AVPacket packet;

@@ -327,6 +335,8 @@
segment_time = prev_segment_time;
}

+ current_segment_duration = (int) round(segment_time - prev_segment_time);
+ actual_segment_durations[last_segment] = (current_segment_duration > 0 ? current_segment_duration: 1);
if (segment_time - prev_segment_time >= segment_duration) {
put_flush_packet(oc->pb);
url_fclose(oc->pb);
@@ -340,7 +350,7 @@
}

if (write_index) {
- write_index = !write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, ++last_segment, 0, max_tsfiles);
+ write_index = !write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, ++last_segment, 0, max_tsfiles, actual_segment_durations);
}

if (remove_file) {
@@ -372,7 +382,9 @@

av_write_trailer(oc);

- avcodec_close(video_st->codec);
+ if (video_index >= 0) {
+ avcodec_close(video_st->codec);
+ }

for(i = 0; i < oc->nb_streams; i++) {
av_freep(&oc->streams[i]->codec);
@@ -391,7 +403,7 @@
}

if (write_index) {
- write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, ++last_segment, 1, max_tsfiles);
+ write_index_file(index, tmp_index, segment_duration, output_prefix, http_prefix, first_segment, ++last_segment, 1, max_tsfiles, actual_segment_durations);
}

if (remove_file) {
@@ -399,6 +411,8 @@
remove(remove_filename);
}

+ free(actual_segment_durations);
+
return 0;
}



I had assumed that the resets were triggered by Comcast deliberately as a way of discouraging their users from generating upstream traffic (i.e. file sharing). It suffices to say that calling Comcast technical support was less than helpful.

I accessed the diagnostics page for my Motorola modem (http://192.168.100.1/) to see if any useful information might be gleaned. It appeared that everything was normal:

Downstream Value Frequency 717000000 Hz Signal to Noise Ratio 36 dB Downstream Modulation QAM256 Network Access Control Object ON Power Level 2 dBmV

Upstream Value
Channel ID 1
Frequency 30600000 Hz
Ranging Service ID 2379
Symbol Rate 5.120 Msym/s
Power Level 46 dBmV
Upstream Modulation [3] QPSK
[3] 64QAM

Signal Stats Value
Total Unerrored Codewords 2619232444
Total Correctable Codewords 6
Total Uncorrectable Codewords 0

I then began to tinker with the Maximum Transmission Unit (MTU) setting on my wireless router. I have always left this configuration setting to "Auto", which usually results in an MTU value of 1500 bytes. Lo and behold, adjusting the MTU to a lower value has resulted in a significantly more stable connection. I am currently using an MTU value of 1300 bytes, which is fairly low. I tested the stability of my connection with some of broadband speed test websites, such as SpeedTest.Net.

The MTU value specifies the maximum packet size that the router will transmit. When a router receives a packet that exceeds the MTU, it must either fragment the packet into 2 or more smaller packets that meet the MTU requirements, or it must respond to the sender of the large packet with a message indicating that the packet was too large. Using a low MTU value can lead to more packets which results in more communication overhead. For this reason I might increase the MTU in the future. But in my experience, a lower MTU value on my Comcast connection has led to a greater stability which is definitely preferred.

I've used the Open-Source FFMPEG-based segmenter to produce segmented encodings in the MythPodcaster project. I made some minor changes to the segmenter so that it can segment audio-only input files.

What all of this means is that I can have audio and video encodings at various bitrates for all of my DVR recordings and play them on demand with a quality level that is appropriate for the network conditions (high for WiFi access, medium or low for 3G). The current state of wireless and mobile video technology is truly amazing!

Download the updated build from the MythPodcaster site.

But I excitement quickly changed to disappointment upon learning that Comcast has applied 5C content protection to their HD channels, which means that unauthorized devices (such as our MythTV) are not authorized to record or view the content via the Firewire port. This means that our HD-capable cable-box is effectively no different from a cheaper standard-definition cable-box. I've chosen to record HD content from a broadcast antenna, which generally offers better video quality at no cost. But it's frustrating that Comcast and other cable providers offer HD with the caveat that it can only be viewed with select devices. It's a total bait-and-switch.

Monitoring and tracking our activities is becoming incredibly popular. WIRED Magazine recently had an issue with the title 'Living by Numbers' in which they described many popular applications and products that enable people to quantify the details of their life. There are huge benefits to knowing more about how we live, but systems that are involved must be designed with privacy in mind.

But with more data comes more problems. What if the details of your body temperature, diet, or geographic movements find their way into the hands of a prospective employer, insurance agent, or any individual or entity who may not have your best interests at heart? Securing that data is paramount. And I think the easiest way to secure that data is to own it in its entirety. Cloud Computing (GMail, Google Documents, Twitter, Facebook, MySpace, etc) may make it easy to access, share, and compare your data, but it provides control and ownership of that data to an external entity whose security measures are a complete unknown.

So, I think that it's very important that data related your day-to-day activities be kept as close to you, the rightful owner, as possible. As we design and develop tools to capture and record this data, we must provide additional tools that allow the owners to access and transfer their data as they see fit.

But there has been one problem that occurs intermittently: the audio on live & recorded programs will sound like it's being projected from a tin can. The sound is discernible, but wrong. Enduring an hour-long program with bad sound can be difficult.

At first, I wasn't sure if the sound distortion was coming from the digital cable box, the PVR-150 encoder card, or MythTV playback. We had never experienced this kind of audio distortion with our old Tivo service using the same digital cable box, so I had to rule out the cable box. Also, we never had any problems when using the PVR-150 tuner input, which combines video & audio in a single input. I had recently switched from the tuner input to the composite audio & video input on the PVR-150 due to the generally higher quality they deliver in comparison to coaxial feeds. So, I concluded that problem was with the PVR-150 composite audio input.

I'd noticed that leaving and then promptly returning to the channel would resolve the problem. But this is not possible when recording programs while away. After reading several posting on MythTV user lists, I decided to modify the channel-change script to fork a background that re-establishes the audio input 2 seconds after the channel-change script returns. This has worked successfully for a week now, which is a marked improvement over where we were.

The following shows the channel-change script:

[scott@eowyn ~]$ cat /usr/local/bin/change_chan.sh 
#!/bin/sh

# send infrared signals to change the current channel
REMOTE_NAME=comcast
for digit in $(echo $1 | sed -e 's/./& /g'); do 
  /usr/local/bin/irsend SEND_ONCE $REMOTE_NAME $digit
  sleep 0.4
done
/usr/local/bin/irsend SEND_ONCE $REMOTE_NAME "ok"

# start audio fix script in the background
/usr/local/bin/ivtv_audio_fix.sh &

And the following shows the contents of the audio fix-it script that runs in the background after the channel-change process has finished.

[scott@eowyn ~]$ cat /usr/local/bin/ivtv_audio_fix.sh 
#!/bin/sh

sleep 2

# reset the audio input to source 1
/usr/local/bin/v4l2-ctl --set-audio-input 1

I'll also mention that I don't notice any glitches in the audio despite the fact that the fix-it script runs 2 seconds after the channel has been changed. ]]> http://urlgrey.net/archives/2008/06/audio_distortio.php http://urlgrey.net/archives/2008/06/audio_distortio.php MythTV Sat, 07 Jun 2008 16:16:43 -0800 I had enabled Open Vsync on our MythTV frontend during the initial configuration with the belief that there couldn't be any harm in adding an additional method of vertical synchronization. I changed my mind when I began noticing strange delays in the video playback on my NVidia FX 5200 with XVMC. After changing channels, there would be about 3-5 glitches in video playback for 30 seconds, followed by okay playback. I was okay with it until we began seeing much more dramatic choppiness in playback coupled with higher CPU utilization (typically 18-20%, but as high as 70%).

I searched the web for the symptoms and saw a lot of people reporting problems with video playback when OpenGL Vsync is enabled (here and here). Since disabling OpenGL Vsync, I've not seen any delays in playback. I might re-enable UseEvents in /etc/X11/xorg.conf, but it's currently set to False.

I found the XvMC section of the MythTV wiki describing how to use X-Video Motion Compensation (XvMC) to utilize hardware acceleration when decoding MPEG-2 video. It actually took me several hours to get XvMC working, but I'll summarize the effective steps below:

First, install the NVidia display driver for Linux. The x.org driver doesn't support XvMC, so there's no way around it. My previous post has information on how to do this.

First, set-up the /etc/X11/xorg.conf file for the NVidia driver. XvMC worked only when NVAGP was set to '1'. You find explanations of these options on the NVidia support site. The relevant section of my /etc/X11/xorg.conf follows:

Section "Device"
    Identifier     "Videocard0"
    Driver         "nvidia"
    Option  "NVAGP"                 "1"
    Option  "NoLogo"                "True"
    Option  "RenderAccel"           "True"
    Option  "XvmcUsesTextures"      "True"
    Option  "UseEdidDpi"            "False"
    Option  "DPI"                   "100 x 100"
    Option  "UseEvents"             "False"
    Option  "DPMS"                  "False"
EndSection

Next, create or edit the file /etc/X11/XvMCConfig. Following are some shell commands that show my system setup:

[scott@eowyn lib]$ cd /usr/lib
[scott@eowyn lib]$ ll libXvMCNVIDIA*
-r--r--r-- 1 root root 152808 2008-03-31 08:47 libXvMCNVIDIA.a
lrwxrwxrwx 1 root root     23 2008-03-31 08:47 libXvMCNVIDIA_dynamic.so.1 -> libXvMCNVIDIA.so.169.12
-rwxr-xr-x 1 root root 139604 2008-03-31 08:47 libXvMCNVIDIA.so.169.12
[scott@eowyn lib]$ cat /etc/X11/XvMCConfig 
libXvMCNVIDIA_dynamic.so.1

To confirm that XvMC is working in X Windows, start up X Windows (i.e. startx), create a shell window and run xdpyinfo to look for the XvMC info:

[scott@eowyn lib]$ xdpyinfo | grep XVideo
     XVideo
     XVideo-MotionCompensation

The next step is to instruct mythfrontend to use XvMC for playback. I did this by choosing a playback profile in the mythfrontend via the Setup -> TV Settings -> Playback menus. The CPU-- profile uses 'ivtv' for Standard Definition (SD) Television, and 'xvmc' for higher definition content. I'm only recording SD content right now, but would like to use 'xvmc' to lower the CPU load. So, I created a new playback profile that uses 'xvmc' for all content.

I'll also mention that the picture became skewed on my 16:9 Sony 32" flat-panel television. I corrected this by adding scaling values of X=2, Y=3 in the Setup -> TV Settings -> Playback menu. This shifted the picture left and down to fill the screen. I'm also using the DVI output of the NVidia FX 5200 card. I've got a DVI-to-HDMI cable that delivers a pure digital signal to the television. The clarity is phenomenal.

After making all of these changes, I was able to confirm that XvMC was in use simply by looking at the CPU utilization of the 'mythfrontend' process with 'top'. Before I enabled XvMC, CPU utilization was at 68-72% when playing video. After switching to XvMC, utilization is merely 18-20%. That's more than a 300% reduction in CPU utilization! Since I use the MythTV machine for the backend and frontend, the reduction in CPU utilization opens up the possibility of adding another encoder card, or running some additional services such as ZoneMinder. ]]> http://urlgrey.net/archives/2008/04/using_xvmc_with.php http://urlgrey.net/archives/2008/04/using_xvmc_with.php MythTV Tue, 01 Apr 2008 11:16:04 -0800 While fine-tuning our MythTV system, I noticed that the default menu-renderer is QT. There's nothing wrong with QT, except that it doesn't offer hardware acceleration that OpenGL does. So, using QT creates a higher system load on the MythTV frontend. This takes away CPU cycles that could be better spent, and provides a UI that is less smooth.

So, I enabled OpenGL menu-rendering for MythTV (0.21) only to find the menu transition taking a horrendous amount of time. There was an OpenGL fade that occurred on each screen transition that would take about 5 second to complete. This is when I realized that I should have installed the vendor's X Windows driver.

My MythTV machine has an NVidia GeForce FX 5200 graphics card installed. It's a few years old, but works perfectly for this purpose: fanless, hardware OpenGL rendering & MPEG-2 decoding, and DVI output. I went to the NVidia support site and downloaded the display driver. I then installed it and marveled at how much it improved everything. The system load was greatly reduced because the OpenGL rendering occurred in hardware, menu navigation was very quick, and a lovely fade effect occurs upon each screen transition.

So, the lesson learned was to install the NVidia display drivers when choosing OpenGL menu rendering with MythTV. OpenGL is a great improvement over QT, and having display drivers that can take advantage of hardware acceleration is a must.