Boxing Glove (Accelerometer Basics)

Boxing Glove is an app for people who feel like being immature (or people who simply are immature). With it, you can throw punches into the air and hear a variety of punching/groaning sound effects. The punching sound effects occur right when you hit your imaginary target.

Boxing Glove supports right- or left-handed punching, and has a few entertaining features, such as a button that makes a “ding ding ding” bell sound as if to signify the start of a fight.

To provide the effect of making a punching sound at the appropriate time, this app uses the phone’s accelerometer.

The Accelerometer

Several times a second, the phone’s accelerometer reports the direction and magnitude of the total force being applied to the phone. This force is expressed with three values—X, Y, and Z—where X is horizontal, Y is vertical, and Z is perpendicular to the screen. This is illustrated in Figure 44.1.

The three accelerometer dimensions, relative to the phone screen.
FIGURE 44.1 The three accelerometer dimensions, relative to the phone screen.

The magnitude of each value is a multiplier of g (the gravitational force on the surface of Earth). Each value is restricted to a range from -2 to 2. If the phone is resting flat on a table with the screen up, the values reported for X and Y are roughly zero, and the value of Z is roughly -1 (1 g into the screen toward the ground). That’s because the only force being applied to the phone in this situation is gravity. By shifting the phone’s angle and orientation and then keeping it roughly still, the values of X, Y, and Z reveal which way is down in the real world thanks to the ever-present force of gravity. When you abruptly move or shake the phone, the X, Y, and Z values are able to reveal this activity as well.

Regardless of how you contort your phone, the X,Y, and Z axes used for the accelerometer data remain fixed to the phone. For example, the Y axis always points toward the top edge of the phone.

To get the accelerometer data, you create an instance of the Accelerometer class from the Microsoft.Devices.Sensors namespace in the Microsoft.Devices.Sensors assembly.

This assembly is not referenced by Windows Phone projects by default, so you must add it via the Add Reference dialog in Visual Studio.

The Accelerometer class exposes Start and Stop methods and—most importantly— a ReadingChanged event. This event gets raised many times a second (after Start is called) and reports the data via properties on the event-args parameter passed to handlers. These properties are X, Y, Z, and Timestamp. The physical accelerometer is always running; Start and Stop simply start/stop the data reporting to your app.

Sounds pretty simple, right? The class is indeed simple although, as you’ll see in the remaining chapters, interpreting the data in a satisfactory way can be complicated.

To get the best performance and battery life, it’s good to stop the accelerometer data reporting when you don’t need the data and then restart it when you do.

The main page, with its application bar menu expanded.
FIGURE 44.2 The main page, with its application bar menu expanded.

The User Interface

Boxing Glove has a main page, a settings page, an instructions page, and an about page. The latter two pages aren’t interesting and therefore aren’t shown in this chapter, but Listing 44.1 contains the XAML for the main page. The page, with its application bar menu expanded, is shown in Figure 44.2.

LISTING 44.1 MainPage.xaml—The User Interface for Boxing Glove’s Main Page

[code]

<phone:PhoneApplicationPage
x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
SupportedOrientations=”Portrait”>
<!– The application bar, with two buttons and three menu items –>
<phone:PhoneApplicationPage.ApplicationBar>
<shell:ApplicationBar Opacity=”.5”>
<shell:ApplicationBarIconButton Text=”ring bell”
IconUri=”/Images/appbar.bell.png” Click=”RingBellButton_Click” />
<shell:ApplicationBarIconButton Text=”switch hand”
IconUri=”/Images/appbar.leftHand.png”
Click=”SwitchHandButton_Click” />
<shell:ApplicationBar.MenuItems>
<shell:ApplicationBarMenuItem Text=”instructions”
Click=”InstructionsMenuItem_Click”/>
<shell:ApplicationBarMenuItem Text=”settings”
Click=”SettingsMenuItem_Click”/>
<shell:ApplicationBarMenuItem Text=”about”
Click=”AboutMenuItem_Click”/>
</shell:ApplicationBar.MenuItems>
</shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>
<Border Background=”{StaticResource PhoneAccentBrush}”>
<Image Source=”Images/hand.png” RenderTransformOrigin=”.5,.5”>
<Image.RenderTransform>
<!– ScaleX is 1 for right-handed or -1 for left-handed –>
<CompositeTransform x:Name=”ImageTransform” ScaleX=”1”/>
</Image.RenderTransform>
</Image>
</Border>
</phone:PhoneApplicationPage>

[/code]

Besides the application bar, this page basically contains an image that instructs the user how to hold the phone. The background takes on the theme accent color, so the screen in Figure 44.2 is from a phone whose accent color is set to red.

The application bar contains a button for performing a “ding ding ding” bell sound on demand (to mimic the start of a fight in a boxing ring), and a button for swapping between right-handed mode and left-handed mode. The composite transform flips the image horizontally (in code-behind) when left-handed mode is in use.

The Code-Behind

Listing 44.2 contains the code-behind for the main page. It makes use of two persisted settings defined in a separate Settings.cs file as follows:

[code]

public static class Settings
{
public static readonly Setting<bool> IsRightHanded =
new Setting<bool>(“IsRightHanded”, true);
public static readonly Setting<double> Threshold =
new Setting<double>(“Threshold”, 1.5);
}

[/code]

LISTING 44.2 MainPage.xaml.cs—The Code-Behind for Boxing Glove’s Main Page

[code]

using System;
using System.Windows;
using System.Windows.Media;
using System.Windows.Navigation;
using Microsoft.Devices.Sensors;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Shell;
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
Accelerometer accelerometer;
IApplicationBarIconButton switchHandButton;
DateTimeOffset acceleratingQuicklyForwardTime = DateTimeOffset.MinValue;
Random random = new Random();
double currentThreshold;
public MainPage()
{
InitializeComponent();
this.switchHandButton = this.ApplicationBar.Buttons[1]
as IApplicationBarIconButton;
// Initialize the accelerometer
this.accelerometer = new Accelerometer();
this.accelerometer.ReadingChanged += Accelerometer_ReadingChanged;
SoundEffects.Initialize();
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
// Start the accelerometer
try
{
this.accelerometer.Start();
}
catch
{
MessageBox.Show(
“Unable to start your accelerometer. Please try running this app again.”,
“Accelerometer Error”, MessageBoxButton.OK);
}
// Also ensures the threshold is updated on return from settings page
UpdateForCurrentHandedness();
}
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
base.OnNavigatedFrom(e);
// Stop the accelerometer
try
{
this.accelerometer.Stop();
}
catch { /* Nothing to do */ }
}
// Process data coming from the accelerometer
void Accelerometer_ReadingChanged(object sender,
AccelerometerReadingEventArgs e)
{
// Only pay attention to large-enough magnitudes in the X dimension
if (Math.Abs(e.X) < Math.Abs(this.currentThreshold))
return;
// See if the force is in the same direction as the threshold
// (forward punching motion)
if (e.X * this.currentThreshold > 0)
{
// Forward acceleration
this.acceleratingQuicklyForwardTime = e.Timestamp;
}
else if (e.Timestamp – this.acceleratingQuicklyForwardTime
< TimeSpan.FromSeconds(.2))
{
// This is large backward force shortly after the forward force.
// Time to make the punching noise!
this.acceleratingQuicklyForwardTime = DateTimeOffset.MinValue;
// We’re on a different thread, so transition to the UI thread.
// This is a requirement for playing the sound effect.
this.Dispatcher.BeginInvoke(delegate()
{
switch (this.random.Next(0, 4))
{
case 0: SoundEffects.Punch1.Play(); break;
case 1: SoundEffects.Punch2.Play(); break;
case 2: SoundEffects.Punch3.Play(); break;
case 3: SoundEffects.Punch4.Play(); break;
}
switch (this.random.Next(0, 10)) // Only grunt some of the time
{
case 0: SoundEffects.Grunt1.Play(); break;
case 1: SoundEffects.Grunt2.Play(); break;
case 2: SoundEffects.Grunt3.Play(); break;
}
});
}
}
void UpdateForCurrentHandedness()
{
this.currentThreshold = (Settings.IsRightHanded.Value ?
Settings.Threshold.Value :
-Settings.Threshold.Value);
this.ImageTransform.ScaleX = (Settings.IsRightHanded.Value ? 1 : -1);
// Show the opposite hand on the application bar button
if (Settings.IsRightHanded.Value)
this.switchHandButton.IconUri = new Uri(“/Images/appbar.leftHand.png”,
UriKind.Relative);
else
this.switchHandButton.IconUri = new Uri(“/Images/appbar.rightHand.png”,
UriKind.Relative);
}
// Application bar handlers
void RingBellButton_Click(object sender, EventArgs e)
{
SoundEffects.DingDingDing.Play();
}
void SwitchHandButton_Click(object sender, EventArgs e)
{
Settings.IsRightHanded.Value = !Settings.IsRightHanded.Value;
UpdateForCurrentHandedness();
}
void InstructionsMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/InstructionsPage.xaml”,
UriKind.Relative));
}
void SettingsMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/SettingsPage.xaml”,
UriKind.Relative));
}
void AboutMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/AboutPage.xaml”,
UriKind.Relative));
}
}
}

[/code]

  • The constructor contains the code for initializing the accelerometer. It constructs an instance of Accelerometer and attaches a handler to its ReadingChanged event. The SoundEffects class, defined in Listing 44.3, is also initialized.
  • OnNavigatedTo contains the code for starting the accelerometer, and OnNavigatedTo stops it. If it weren’t stopped, the handler would still be called (and the sound effects would still be made) while the main page is on the back stack. This would happen when the user visits the settings, instructions, or about pages. Leaving the accelerometer running would not be a problem, and may even be desirable for some apps. However, Listing 44.1 stops it to avoid two threads potentially reading/writing the threshold variable at the same time, as discussed in a later sidebar.

The calls to Accelerometer.Start and Accelerometer.Stop can throw an exception!

This can happen at development-time if you omit the ID_CAP_SENSORS capability from your app manifest. For a published app in the marketplace (which automatically gets the appropriate capability), this should not happen unless you have previously called the Accelerometer instance’s Dispose method.Of course, there’s not much that many accelerometer-based apps can do when the accelerometer fails to start, so Boxing Glove simply instructs to user to try closing and reopening the app and hope for the best.

  • Inside Accelerometer_ReadingChanged, the handler for the ReadingChanged event, only two properties of the AccelerometerReadingEventArgs instance are examined: X and Timestamp. The algorithm is as follows: If the app detects a strong forward horizontal force followed quickly by a strong backward horizontal force, it’s time to make a punching sound. Making the sound when the forward force is detected isn’t good enough, because the sound would be made too early. The sound should occur when the punching motion stops (i.e. hits the imaginary target of the punch). The detected backward force does not result from the phone being moved backward, but rather from the fast deceleration that occurs when the user stops their flying fist in mid-air.
  • The definition of “strong” used by Accelerometer_ReadingChanged’s algorithm is determined by a threshold that is configurable by the user on the settings page. The absolute value of the threshold ranges from almost 0 (.1) to almost 2 (1.9). The sign of the threshold depends on whether the app is in right-handed mode or lefthanded mode. In right-handed mode, forward motion means pushing to phone toward its left, so the threshold of forward force is negative. In left-handed mode, forward motion means pushing the phone toward its right, so the threshold of forward force is positive. This adjustment is made inside UpdateForCurrentHandedness.
  • When it’s time to make a punching noise, the sound is randomly chosen, potentially along with a randomly chosen grunting sound.

The accelerometer’s ReadingChanged event is raised on a non-UI thread!

This is great for processing the data without creating a bottleneck on the UI thread, but it does mean that you must explicitly transition to the UI thread before performing any work that requires it.This includes updating any UI elements or, as in this app, playing a sound effect. Listing 44.2 uses the page dispatcher’s BeginInvoke method to play the sound effect on the UI thread.

Although you can start the accelerometer from a page’s constructor, it’s normally better to wait until an event such as Loaded.That’s because starting it within the constructor could cause the ReadingChanged event to be raised earlier than when you’re prepared to handle it. For example, it can be raised before (or during) the deserialization of persisted settings, causing a failure if a setting is accessed in the event handler. It can be raised before the CompositionTarget.Rendering event is raised,which would be a problem if the implementation of SoundEffects (shown in the next listing) waited for the first Rendering event to call XNA’s FrameworkDispatcher.Update method.

Boxing Glove chooses to start the accelerometer in OnNavigatedTo and stop it in OnNavigatedFrom so the ReadingChanged event-handler thread doesn’t try to access the threshold member while the call to UpdateForCurrentHandedness inside OnNavigatedTo potentially updates it on the UI thread.

The SoundEffects class used by Listing 44.2 is shown in Listing 44.3. It encapsulates the work of setting up the sound effects, with code similar to the apps from Part V, “Audio & Video.”

LISTING 44.3 SoundEffects.cs—Initializes and Exposes Boxing Glove’s Eight Sound Effects

[code]

using System;
using System.Windows.Resources;
using Microsoft.Xna.Framework.Audio; // For SoundEffect
namespace WindowsPhoneApp
{
public static class SoundEffects
{
public static void Initialize()
{
StreamResourceInfo info;
info = App.GetResourceStream(new Uri(“Audio/punch1.wav”, UriKind.Relative));
Punch1 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/punch2.wav”, UriKind.Relative));
Punch2 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/punch3.wav”, UriKind.Relative));
Punch3 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/punch4.wav”, UriKind.Relative));
Punch4 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/grunt1.wav”, UriKind.Relative));
Grunt1 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/grunt2.wav”, UriKind.Relative));
Grunt2 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/grunt3.wav”, UriKind.Relative));
Grunt3 = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(new Uri(“Audio/dingDingDing.wav”,
UriKind.Relative));
DingDingDing = SoundEffect.FromStream(info.Stream);
CompositionTarget.Rendering += delegate(object sender, EventArgs e)
{
// Required for XNA Sound Effect API to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
};
// Call also once at the beginning
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
}
public static SoundEffect Punch1 { get; private set; }
public static SoundEffect Punch2 { get; private set; }
public static SoundEffect Punch3 { get; private set; }
public static SoundEffect Punch4 { get; private set; }
public static SoundEffect Grunt1 { get; private set; }
public static SoundEffect Grunt2 { get; private set; }
public static SoundEffect Grunt3 { get; private set; }
public static SoundEffect DingDingDing { get; private set; }
}
}

[/code]

The Settings Page

The settings page, shown in Figure 44.3, contains a slider and a reset button for adjusting the threshold value from .1 to 1.9. The XAML is shown in Listing 44.4 and its codebehind is in Listing 44.5.

The settings page enables the user to adjust the accelerometer threshold, described as “required punching strength.”
FIGURE 44.3 The settings page enables the user to adjust the accelerometer threshold, described as “required punching strength.”

LISTING 44.4 SettingsPage.xaml—The User Interface for Boxing Glove’s Settings Page

[code]

<phone:PhoneApplicationPage
x:Class=”WindowsPhoneApp.SettingsPage” x:Name=”Page”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
xmlns:local=”clr-namespace:WindowsPhoneApp”
FontFamily=”{StaticResource PhoneFontFamilyNormal}”
FontSize=”{StaticResource PhoneFontSizeNormal}”
Foreground=”{StaticResource PhoneForegroundBrush}”
SupportedOrientations=”PortraitOrLandscape”
shell:SystemTray.IsVisible=”True”>
<Grid Background=”{StaticResource PhoneBackgroundBrush}”>
<Grid.RowDefinitions>
<RowDefinition Height=”Auto”/>
<RowDefinition Height=”*”/>
</Grid.RowDefinitions>
<!– The standard settings header –>
<StackPanel Grid.Row=”0” Style=”{StaticResource PhoneTitlePanelStyle}”>
<TextBlock Text=”SETTINGS” Style=”{StaticResource PhoneTextTitle0Style}”/>
<TextBlock Text=”boxing glove”
Style=”{StaticResource PhoneTextTitle1Style}”/>
</StackPanel>
<ScrollViewer Grid.Row=”1”>
<StackPanel Margin=”{StaticResource PhoneMargin}”>
<TextBlock Text=”Required punching strength”
Foreground=”{StaticResource PhoneSubtleBrush}”
Margin=”{StaticResource PhoneMargin}”/>
<Slider x:Name=”StrengthSlider” Minimum=”.1” Maximum=”1.9”
LargeChange=”.1”
Value=”{Binding Threshold, Mode=TwoWay, ElementName=Page}”/>
<Button Content=”reset” Click=”ResetButton_Click”
local:Tilt.IsEnabled=”True”/>
<TextBlock TextWrapping=”Wrap” Margin=”{StaticResource PhoneMargin}”
Text=”…”/>
</StackPanel>
</ScrollViewer>
</Grid>
</phone:PhoneApplicationPage>

[/code]

LISTING 44.5 SettingsPage.xaml.cs—The Code-Behind for Boxing Glove’s Settings Page

[code]

using System.Windows;
using Microsoft.Phone.Controls;
namespace WindowsPhoneApp
{
public partial class SettingsPage : PhoneApplicationPage
{
public SettingsPage()
{
InitializeComponent();
}
// Simple property bound to the slider
public double Threshold
{
get { return Settings.Threshold.Value; }
set { Settings.Threshold.Value = value; }
}
void ResetButton_Click(object sender, RoutedEventArgs e)
{
this.StrengthSlider.Value = Settings.Threshold.DefaultValue;
}
}
}

[/code]

The Finished Product

Boxing Glove (Accelerometer Basics)

Talking Parrot (Recording & Playing)

The Talking Parrot app provides more entertainment with the microphone. After greeting you with a friendly “hello,” the parrot listens to what you say and then repeats it in its own voice, with a whistle or squawk thrown in. This app must not only turn the buffer collected from the microphone into a playable XNA sound effect, but it also must determine when is a good time to listen to the user and when is a good time to play the captured audio.

The Main User Interface

Listing 35.1 contains the XAML for the main page. It consists of three parts: a bunch of animations that are triggered by code-behind, an application bar, and several images (plus one ellipse) placed in specific spots on a canvas. The individual images that form the parrot are shown in Figure 35.1.

The parrot consists of nine images that can be individually animated.
FIGURE 35.1 The parrot consists of nine images that can be individually animated.

LISTING 35.1 MainPage.xaml—The User Interface for Talking Parrot’s Main Page

[code]

<phone:PhoneApplicationPage x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
SupportedOrientations=”Portrait”>
<!– Animations as named resources –>
<phone:PhoneApplicationPage.Resources>
<Storyboard x:Name=”BlinkStoryboard” Duration=”0:0:4” RepeatBehavior=”Forever”
Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”TopEyelidImage” To=”0”
Duration=”0:0:.1” AutoReverse=”True”/>
<DoubleAnimation Storyboard.TargetName=”BottomEyelidImage” To=”77”
Duration=”0:0:.1” AutoReverse=”True”/>
</Storyboard>
<Storyboard x:Name=”HeadUpStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”HeadCanvas” To=”-7”
Duration=”0:0:1”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”HeadDownStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”HeadCanvas” To=”0” Duration=”0:0:1”>
<DoubleAnimation.EasingFunction>
<QuadraticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”WingFlutterStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.Rotation)”>
<DoubleAnimation Storyboard.TargetName=”WingImage” To=”35” Duration=”0:0:1”
AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<BounceEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”StompStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”LeftFootImage” To=”-30”
Duration=”0:0:.2” AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
<DoubleAnimation Storyboard.TargetName=”RightFootImage” To=”-10”
BeginTime=”0:0:.1” Duration=”0:0:.2” AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”SpeakStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.Rotation)”>
<DoubleAnimationUsingKeyFrames x:Name=”TopBeakAnimation”
Storyboard.TargetName=”TopBeakImage” />
<DoubleAnimationUsingKeyFrames x:Name=”BottomBeakAnimation”
Storyboard.TargetName=”BottomBeakImage” />
</Storyboard>
</phone:PhoneApplicationPage.Resources>
<!– The application bar –>
<phone:PhoneApplicationPage.ApplicationBar>
<shell:ApplicationBar Opacity=”.5”>
<shell:ApplicationBarIconButton Text=”instructions”
IconUri=”/Shared/Images/appbar.instructions.png”
Click=”InstructionsButton_Click”/>
<shell:ApplicationBar.MenuItems>
<shell:ApplicationBarMenuItem Text=”settings”
Click=”SettingsMenuItem_Click”/>
<shell:ApplicationBarMenuItem Text=”about”
Click=”AboutMenuItem_Click”/>
</shell:ApplicationBar.MenuItems>
</shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>
<!– Many images placed in a canvas –>
<Canvas>
<Image Source=”Images/background.png” Stretch=”None”/>
<Image x:Name=”RightFootImage” Source=”Images/rightFoot.png”
Canvas.Left=”159” Canvas.Top=”537”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”LeftFootImage” Source=”Images/leftFoot.png”
Canvas.Left=”109” Canvas.Top=”532”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Image.RenderTransform>
</Image>
<Image Source=”Images/body.png” Canvas.Left=”-164” Canvas.Top=”346”/>
<Canvas x:Name=”HeadCanvas”>
<Canvas.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Canvas.RenderTransform>
<Image Source=”Images/head.png” Canvas.Left=”155” Canvas.Top=”203”/>
<Image x:Name=”BottomBeakImage” Source=”Images/bottomBeak.png”
Canvas.Left=”282” Canvas.Top=”294” RenderTransformOrigin=”.5,0”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”15” CenterY=”15”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”TopBeakImage” Source=”Images/topBeak.png”
Canvas.Left=”279” Canvas.Top=”252”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”38” CenterY=”38”/>
</Image.RenderTransform>
</Image>
<Image Source=”Images/eyeball.png” Canvas.Left=”198” Canvas.Top=”248”/>
<Ellipse x:Name=”Pupil” Fill=”Black” Width=”18” Height=”18”
Canvas.Left=”240” Canvas.Top=”275”/>
<Image x:Name=”AngryEyelidImage” Source=”Images/eyelid.png”
Visibility=”Collapsed” Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform Rotation=”15” CenterX=”76” CenterY=”39”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”TopEyelidImage” Source=”Images/eyelid.png”
Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”-37” Rotation=”0”
CenterX=”76” CenterY=”39”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”BottomEyelidImage” Source=”Images/eyelid.png”
Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform ScaleY=”-1” TranslateY=”112”/>
</Image.RenderTransform>
</Image>
</Canvas>
<Image x:Name=”WingImage” Source=”Images/wing.png”
Canvas.Left=”68” Canvas.Top=”414”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”190” CenterY=”26”/>
</Image.RenderTransform>
</Image>
</Canvas>
</phone:PhoneApplicationPage>

[/code]

  • The page is portrait-only due to the exact layout required by the background and pieces of the parrot.
  • The animations are given names rather than dictionary keys, so they can be easily referenced from code-behind.
  • The animations lower, raise, and rotate various pieces of the parrot when triggered by code-behind. Every animatable piece uses a composite transform for consistency and for ease in animating multiple aspects (such as rotation and translation).
  • SpeakStoryboard is special. Although it has two keyframe animations to animate the top and bottom beak, they start out empty. These are continually updated from code-behind based on the audio to be spoken, so the parrot appears to mouth the actual words and sounds it speaks.
  • Any of the parrot pieces (as well as the background) could have been created as vector shapes instead of images. Instead, the black pupil is the only vector shape used in the parrot (an ellipse). This gives us the flexibility to grow/shrink the pupil without pixelation and the flexibility to change its color, although this app doesn’t take advantage of these capabilities.

The Main Code-Behind

The code-behind for the main page is shown in Listing 35.2.

LISTING 35.2 MainPage.xaml.cs—The Code-Behind for Talking Parrot’s Main Page

[code]

using System;
using System.IO;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using Microsoft.Phone.Controls;
using Microsoft.Xna.Framework.Audio;
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
byte[] buffer;
// Used for capturing audio from the microphone
MemoryStream recordedStream;
long playbackStartPosition = -1;
int consecutiveSilentSamples;
DateTime? speakingDoneTime;
// Used for playing the three included sounds: hello, whistle, and squawk
bool playingIncludedSound;
Random random = new Random();
public MainPage()
{
InitializeComponent();
SoundEffects.Initialize();
CompositionTarget.Rendering += CompositionTarget_Rendering;
// Start blinking, which runs the whole time
this.BlinkStoryboard.Begin();
// Prevent the off-screen tail from being seen when
// animating to the instructions or about pages
this.Clip = new RectangleGeometry {
Rect = new Rect(0, 0, Constants.SCREEN_WIDTH, Constants.SCREEN_HEIGHT) };
// Configure the microphone with the smallest supported BufferDuration (.1)
Microphone.Default.BufferDuration = TimeSpan.FromSeconds(.1);
Microphone.Default.BufferReady += Microphone_BufferReady;
// Initialize the buffer for holding microphone data
int size = Microphone.Default.GetSampleSizeInBytes(
Microphone.Default.BufferDuration);
this.buffer = new byte[size];
// Initialize the stream used to record microphone data
this.recordedStream = new MemoryStream();
// Speak a “hello” greeting
PrepareStoryboardForIncludedSound(SoundEffects.HelloBuffer);
Speak(SoundEffects.Hello, 0);
}
protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e)
{
base.OnMouseLeftButtonDown(e);
// Get mad!
this.AngryEyelidImage.Visibility = Visibility.Visible;
}
protected override void OnMouseLeftButtonUp(MouseButtonEventArgs e)
{
base.OnMouseLeftButtonUp(e);
// Become happy again
this.AngryEyelidImage.Visibility = Visibility.Collapsed;
}
void CompositionTarget_Rendering(object sender, EventArgs e)
{
// Required for XNA Microphone API to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
// Check if currently-playing audio has finished
if (this.speakingDoneTime != null && DateTime.Now > this.speakingDoneTime)
{
if (!this.playingIncludedSound)
{
// Don’t restart the microphone yet! We just played audio from the
// microphone, so add either a whistle or squawk to make it sound more
// like a parrot. This will reset speakingDoneTime.
int choice = random.Next(2); // A random number: either 0 or 1
byte[] buffer = (choice == 0 ? SoundEffects.WhistleBuffer :
SoundEffects.SquawkBuffer);
SoundEffect effect = (choice == 0 ? SoundEffects.Whistle :
SoundEffects.Squawk);
PrepareStoryboardForIncludedSound(buffer);
Speak(effect, 0); // Play at the normal speed (0)
}
else
{
// Now it’s time to restart the microphone
Microphone.Default.Start();
// Reset state
this.speakingDoneTime = null;
this.playingIncludedSound = false;
// Smoothly return the head to its resting position
this.HeadDownStoryboard.Begin();
}
}
}
void Speak(SoundEffect effect, float speed)
{
// Stop listening, because it is time to talk
Microphone.Default.Stop();
// Determine when the audio will be done playing and it’s time to either
// add a squawk/whistle or restart the microphone.
// The length is halved for microphone-recorded sounds to avoid extra lag
// time seen in practice.
this.speakingDoneTime = DateTime.Now +
TimeSpan.FromTicks((long)(effect.Duration.Ticks *
(speed == 0 ? 1 : speed / 2)));
// Stop any in-progress storyboards
this.SpeakStoryboard.Stop();
this.WingFlutterStoryboard.Stop();
this.HeadUpStoryboard.Stop();
this.StompStoryboard.Stop();
// Start the storyboards
this.SpeakStoryboard.Begin();
this.WingFlutterStoryboard.Begin();
this.HeadUpStoryboard.Begin();
this.StompStoryboard.Begin();
// Play the audio at full volume with the passed-in speed (pitch)
effect.Play(1, speed, 0);
}
// Changes the contents of TopBeakAnimation and BottomBeakAnimation
// to match the audio in the buffer, so the beak appears to speak the sounds
void PrepareStoryboardForIncludedSound(byte[] buffer)
{
ResetSpeakStoryboard();
// Loop through the buffer in 100-millisecond chunks
for (int i = 0; i < buffer.Length;
i += Constants.INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS)
{
// Cast from short to int to prevent -32768 from overflowing Math.Abs
int currentVolume = Math.Abs((int)BitConverter.ToInt16(buffer, i));
// Add a keyframe to the top & bottom beak animations based on the
// current audio level. ANIMATION_ADJUSTMENT is a fudge factor that
// slightly speeds-up the animation to account for lag.
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
(double)i / Constants.INCLUDED_SOUND_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddSpeakKeyFrame(currentVolume, keyTime);
}
// Add the final keyframe 100 ms later that smoothly closes the beak
KeyTime finalKeyTime = TimeSpan.FromSeconds(Math.Max(0, (double)
(buffer.Length + Constants.INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS) /
Constants.INCLUDED_SOUND_BYTES_PER_SECOND));
AddFinalSpeakKeyFrame(finalKeyTime);
// The preceding work was computationally expensive, so it’s time for
// another update before attempting to play the sound
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
this.playingIncludedSound = true;
}
// Stop the storyboard and empty the keyframes in its two animations
void ResetSpeakStoryboard()
{
SpeakStoryboard.Stop();
TopBeakAnimation.KeyFrames.Clear();
BottomBeakAnimation.KeyFrames.Clear();
}
// Position the top and bottom beak based on the current volume.
// A louder volume results in a wider opening.
void AddSpeakKeyFrame(int currentVolume, KeyTime keyTime)
{
// The top beak rotation should always be an angle between 0 and -50
TopBeakAnimation.KeyFrames.Add(new DiscreteDoubleKeyFrame {
KeyTime = keyTime, Value = Math.Max(-50, currentVolume / -15) });
// The bottom beak rotation should always be an angle between 0 and 30
BottomBeakAnimation.KeyFrames.Add(new DiscreteDoubleKeyFrame {
KeyTime = keyTime, Value = Math.Min(30, currentVolume / 15) });
}
// Close the beak
void AddFinalSpeakKeyFrame(KeyTime keyTime)
{
// Use keyframes that do a smooth quintic ease from the previous values
TopBeakAnimation.KeyFrames.Add(new EasingDoubleKeyFrame {
EasingFunction = new QuinticEase(), KeyTime = keyTime, Value = 0 });
BottomBeakAnimation.KeyFrames.Add(new EasingDoubleKeyFrame {
EasingFunction = new QuinticEase(), KeyTime = keyTime, Value = 0 });
}
void Microphone_BufferReady(object sender, EventArgs e)
{
int size = Microphone.Default.GetData(this.buffer);
if (size == 0)
return;
// Unconditionally record the audio data by writing it to the stream
this.recordedStream.Write(this.buffer, 0, size);
int currentVolume = SoundEffects.GetAverageVolume(this.buffer, size);
if (currentVolume > Settings.VolumeThreshold.Value)
{
// The current volume is loud enough to be considered talking
this.consecutiveSilentSamples = 0;
if (this.playbackStartPosition == -1)
{
// Start a new phrase.
// Back up half a second if we’ve got the data, for a smoother result.
this.playbackStartPosition = Math.Max(0, this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS * 5);
ResetSpeakStoryboard();
}
// Add a keyframe to the beak animations based on the current volume
// ANIMATION_ADJUSTMENT is a fudge factor that slightly speeds-up the
// animation to account for lag.
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
Constants.SOUND_SPEED_FACTOR * (this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS
– this.playbackStartPosition) / Constants.MICROPHONE_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddSpeakKeyFrame(currentVolume, keyTime);
}
else
{
// The current volume is NOT loud enough to be considered talking
this.consecutiveSilentSamples++; // 10 times == 1 second
// Check for the end of a spoken phrase. This happens when we’ve got a
// nonnegative playback start position followed by a second (10 samples)
// of silence.
if (this.playbackStartPosition != -1 &&
this.consecutiveSilentSamples == 10)
{
this.consecutiveSilentSamples = 0;
// Add the final keyframe that smoothly closes the beak
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
Constants.SOUND_SPEED_FACTOR * (this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS
– this.playbackStartPosition) / Constants.MICROPHONE_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddFinalSpeakKeyFrame(keyTime);
// Copy the appropriate slice of audio from the recorded stream into
// a buffer
byte[] buffer = new byte[this.recordedStream.Position –
this.playbackStartPosition];
this.recordedStream.Seek(this.playbackStartPosition, SeekOrigin.Begin);
this.recordedStream.Read(buffer, 0, buffer.Length);
// Amplify the recorded audio, as it tends to be softer than desired
if (Settings.VolumeMultiplier.Value > 1)
SoundEffects.AmplifyAudio(buffer, Settings.VolumeMultiplier.Value);
// Reset variables
this.playbackStartPosition = -1;
this.recordedStream.Position = 0;
// Create a new sound effect from the buffer and speak it
SoundEffect effect = new SoundEffect(buffer,
Microphone.Default.SampleRate, AudioChannels.Mono);
Speak(effect, Constants.SOUND_SPEED_FACTOR);
}
}
}
// Application bar handlers
void InstructionsButton_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/InstructionsPage.xaml”,
UriKind.Relative));
}
void SettingsMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/SettingsPage.xaml”,
UriKind.Relative));
}
void AboutMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(
new Uri(“/Shared/About/AboutPage.xaml?appName=Talking Parrot”,
UriKind.Relative));
}
}
}

[/code]

  • The constructor is nearly identical to the preceding chapter’s constructor, although it doesn’t start the microphone right away because the parrot speaks a greeting instead. We never use the microphone while the parrot is speaking because it might hear itself talk and begin an infinite pattern of repeating itself! Other than speaking the greeting, the other additional tasks are initializing the SoundEffects class, shown in Listing 35.3, initializing a stream for recording microphone audio, and starting the blinking animation that continuously moves the parrot’s eyelids for the duration of the app. The microphone is once again given the shortest possible buffer duration, so the app can remain as responsive as possible.
  • The handlers for screen taps (OnMouseLeftButtonDown and OnMouseLeftButtonUp) toggle the visibility of an “angry eyelid” to give the parrot an annoyed appearance when touched. This could be changed to handle taps specifically on the parrot’s body, but handling taps anywhere on the screen is simpler and likely good enough.
  • CompositionTarget_Rendering, besides calling Update, is responsible for taking action each time the parrot is done speaking. It determines when the parrot is done by comparing the current time to speakingDoneTime, a field set later in the code. If the parrot has just finished speaking audio recorded from the microphone, it makes the parrot speak either a whistle or a squawk (randomly chosen), which resets speakingDoneTime to the later time when the sound effect will finish. If the parrot has just finished speaking one of the included sounds (hello, whistle, or squawk), it restarts the microphone, so it can listen for something new to repeat. As with the preceding chapter, all this code continues to run when navigating forward to a different page. However, this is quite handy for this app because it enables you to aurally test the two settings in real-time as you adjust the sliders on the settings page.
  • The Speak method does the important work of stopping the microphone, setting speakingDoneTime, playing the passed-in sound effect at the passed-in speed, and animating the parrot to mouth the words, flutter its wing, raise its head, and stomp its feet. The magic of SpeakStoryboard mouthing the words in the current audio is enabled by manually updating its animation based on the raw audio data that will be played. This is done inside PrepareStoryboardForIncludedSound for the three included sounds and inside Microphone_BufferReady for the audio recorded from the microphone.
  • PrepareStoryboardForIncludedSound grabs a sample from the passed-in buffer at every 100 milliseconds (mimicking the behavior of the microphone event) and adds a keyframe to the top and bottom beak animations based on the volume of each sample. The louder the sound, the wider the beak needs to be open. The included sounds have a different bitrate than audio captured from the microphone, so the mapping of time to bytes in the buffer is handled by constants specific to these sounds. A final keyframe is added at the end to handle smoothly closing the beak. Because the three included sounds never change, the animations for each one could have been precalculated (or cached after the first time). However, this dynamic approach is done for simplicity and consistency with the code for sounds recorded from the microphone. It also means you can swap in different sound files, and things will work as expected.
  • AddSpeakKeyFrame adds each keyframe as a discrete keyframe (meaning no interpolation between each frame). This is reasonable considering the speed at which the frames advance. It manipulates the value of currentVolume to give it an appropriate range for the angle of rotation for each piece of the beak.
  • AddFinalSpeakKeyFrame gives the final keyframe a quintic (power of five) ease from the preceding value to zero, so the beak snaps shut smoothly.
  • Microphone_BufferReady uses the same approach as the preceding chapter to determine the volume of the last .1 seconds of audio. If the audio is loud enough and we haven’t started tracking the audio as a phrase to play back, we start paying attention to the audio by marking the starting position in the recorded stream and adding a keyframe to the beak animations (as done previously with the included sounds). We continue to listen to the audio (and add keyframes to the animations) until there has been a full second of silence, which equates to ten consecutive samples where the average volume was below the threshold.
  • Because human speech gradually ramps up to a volume above the threshold, simply starting playback at the point where the volume is loud enough would cut off the beginning of whatever was spoken and sound strange. Therefore, the starting position in the recorded stream is backed up half a second from the current point when set inside Microphone_BufferReady. This is why the microphone audio is always appended to the stream, regardless of volume.
  • When the end of relevant audio (a second of silence) has been detected inside Microphone_BufferReady, it copies all the data placed into the recorded stream from the chosen starting position onward into a new byte array. Although previous apps have obtained a sound effect from the static SoundEffect.FromStream method, this code—after potentially amplifying the audio—calls a constructor that enables passing in the raw audio data as a byte array. It then plays the dynamic sound effect (along with the appropriate animations) by calling Speak. It chooses a playback speed (pitch) 80% higher than normal (specified by the SOUND_SPEED_FACTOR constant) so it sounds more like a parrot speaking than the original person whose voice was recorded.
  • The default volume threshold used by Talking Parrot is lower than the one used by Bubble Blower. Here are the two settings used by this app, with their default values:[code]
    public static class Settings
    {
    public static readonly Setting<int> VolumeThreshold =
    new Setting<int>(“VolumeThreshold”, 500);
    public static readonly Setting<int> VolumeMultiplier =
    new Setting<int>(“VolumeMultiplier”, 4);
    }
    [/code]

Talking Parrot’s SoundEffects class is similar to the same-named class in previous chapters, but it also exposes the AmplifyAudio and GetAverageVolume methods used by Listing 35.2. Listing 35.3 contains the implementation.

LISTING 35.3 SoundEffects.cs—Exposes the Built-In Sound Effects and Audio Utility Methods

[code]

using System;
using System.IO;
using System.Windows.Resources;
using Microsoft.Xna.Framework.Audio;
namespace WindowsPhoneApp
{
public static class SoundEffects
{
public static SoundEffect Hello { get; private set; }
public static SoundEffect Squawk { get; private set; }
public static SoundEffect Whistle { get; private set; }
public static byte[] HelloBuffer { get; private set; }
public static byte[] SquawkBuffer { get; private set; }
public static byte[] WhistleBuffer { get; private set; }
public static void Initialize()
{
StreamResourceInfo info;
info = App.GetResourceStream(new Uri(“Audio/hello.wav”, UriKind.Relative));
HelloBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Hello = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(
new Uri(“Audio/squawk.wav”, UriKind.Relative));
SquawkBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Squawk = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(
new Uri(“Audio/whistle.wav”, UriKind.Relative));
WhistleBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Whistle = SoundEffect.FromStream(info.Stream);
// Required for XNA Microphone API to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
}
static byte[] GetBytes(Stream stream)
{
byte[] bytes = new byte[stream.Length];
stream.Read(SquawkBuffer, 0, (int)stream.Length);
return bytes;
}
// Make the sound louder by modifying the raw audio samples
public static void AmplifyAudio(byte[] buffer, int multiplier)
{
// Buffer is an array of bytes, but we want to examine each 2-byte value
for (int i = 0; i < buffer.Length; i += 2)
{
int value = BitConverter.ToInt16(buffer, i);
if (value > Settings.VolumeThreshold.Value)
{
// Only amplify samples that are loud enough to not
// be considered background noise
value *= Settings.VolumeMultiplier.Value;
// Make sure the multiplied value stays within bounds
if (value > short.MaxValue)
value = short.MaxValue;
else if (value < short.MinValue)
value = short.MinValue;
// Replace the two bytes with the amplified value
byte[] newValue = BitConverter.GetBytes(value);
buffer[i] = newValue[0];
buffer[i + 1] = newValue[1];
}
}
}
// Returns the average value among the first numBytes in the buffer
public static int GetAverageVolume(byte[] buffer, int numBytes)
{
long total = 0;
// Buffer is an array of bytes, but we want to examine each 2-byte value
for (int i = 0; i < numBytes; i += 2)
{
// Cast from short to int to prevent -32768 from overflowing Math.Abs:
int value = Math.Abs((int)BitConverter.ToInt16(buffer, i));
total += value;
}
return (int)(total / (numBytes / 2));
}
}
}

[/code]

Unlike in past apps, the raw audio data for each sound file is copied into a byte array exposed as a property. Listing 35.2 used these byte arrays to determine the volume over time, just like what is done for audio from the microphone.

The audio captured from the microphone can often be much softer than desired.To combat this, the AmplifyAudio method in Listing 35.3 increases the volume of the recorded microphone audio by manually multiplying the value of each sample in the buffer (if the sample is louder than a threshold, to avoid amplifying background noise). Although the volume of the played-back audio is ultimately limited by the phone’s volume setting, this technique can make the audio surprisingly loud.Of course, the more that the audio is amplified, the more distorted it may sound.

You can play music from the music library while using this app to make the parrot “sing” the song.You just have to pause the music, so the parrot gets the second of silence needed to prompt it to speak! This can be controlled via the top bar that gets displayed while adjusting the phone’s volume, shown in Figure 35.2.

Music can be played and paused while using Talking Parrot.
FIGURE 35.2 Music can be played and paused while using Talking Parrot.

Here are the constants (and read-only fields) used by this app:

[code]

public static class Constants
{
// Screen
public const int SCREEN_WIDTH = 480;
public const int SCREEN_HEIGHT = 800;
public const float SOUND_SPEED_FACTOR = .8f;
public const float ANIMATION_ADJUSTMENT = .1f;
public static readonly long MICROPHONE_BYTES_PER_SECOND =
Microphone.Default.GetSampleSizeInBytes(TimeSpan.FromSeconds(1));
public static readonly long MICROPHONE_BYTES_PER_100_MILLISECONDS =
Constants.MICROPHONE_BYTES_PER_SECOND / 10;
public const int INCLUDED_SOUND_BYTES_PER_SECOND = 141100;
public const int INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS = 14110;
}

[/code]

The Settings Page

The settings page, shown in Figure 35.3, is like the settings page from the Bubble Blower app, but with two sliders instead of one. The first slider adjusts the “parrot voice volume,” which maps to the VolumeMultiplier setting. The second slider is just like the one from Bubble Blower, which maps to the VolumeThreshold setting. It is labeled as “parrot hearing sensitivity” instead of “microphone sensitivity” to be more appropriate to the theme of this app.

The settings page for Talking Parrot enables changing and resetting VolumeMultiplier and VolumeThreshold.
FIGURE 35.3 The settings page for Talking Parrot enables changing and resetting VolumeMultiplier and VolumeThreshold.

The XAML for Figure 35.3 is shown in Listing 35.4. The differences from Bubble Blower’s settings page are emphasized.

LISTING 35.4 SettingsPage.xaml—The Settings User Interface for Talking Parrot

[code]

<phone:PhoneApplicationPage x:Name=”Page”
x:Class=”WindowsPhoneApp.SettingsPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
xmlns:local=”clr-namespace:WindowsPhoneApp”
FontFamily=”{StaticResource PhoneFontFamilyNormal}”
FontSize=”{StaticResource PhoneFontSizeNormal}”
Foreground=”{StaticResource PhoneForegroundBrush}”
SupportedOrientations=”PortraitOrLandscape”
shell:SystemTray.IsVisible=”True”>
<Grid Background=”{StaticResource PhoneBackgroundBrush}”>
<Grid.RowDefinitions>
<RowDefinition Height=”Auto”/>
<RowDefinition Height=”*”/>
</Grid.RowDefinitions>
<!– The standard settings header: –>
<StackPanel Grid.Row=”0” Style=”{StaticResource PhoneTitlePanelStyle}”>
<TextBlock Text=”SETTINGS” Style=”{StaticResource PhoneTextTitle0Style}”/>
<TextBlock Text=”talking parrot”
Style=”{StaticResource PhoneTextTitle1Style}”/>
</StackPanel>
<!– Stacked contents inside a ScrollViewer,
for the benefit of landscape orientation: –>
<ScrollViewer Grid.Row=”1”>
<StackPanel Margin=”{StaticResource PhoneMargin}”>
<TextBlock Text=”Parrot voice volume”
Foreground=”{StaticResource PhoneSubtleBrush}”
Margin=”{StaticResource PhoneMargin}”/>
<Slider x:Name=”VolumeSlider” Minimum=”1” Maximum=”18”
Value=”{Binding Volume, Mode=TwoWay, ElementName=Page}”/>
<TextBlock Text=”Parrot hearing sensitivity”
Foreground=”{StaticResource PhoneSubtleBrush}”
Margin=”{StaticResource PhoneMargin}”/>
<Slider x:Name=”SensitivitySlider” Maximum=”1000” LargeChange=”100”
IsDirectionReversed=”True”
Value=”{Binding Threshold, Mode=TwoWay, ElementName=Page}”/>
<Button Content=”reset” Click=”ResetButton_Click”
local:Tilt.IsEnabled=”True”/>
</StackPanel>
</ScrollViewer>
</Grid>
</phone:PhoneApplicationPage>

[/code]

  • Because the page is now a little too tall for the landscape orientations, the StackPanel is wrapped inside a ScrollViewer.
  • The allowed range for VolumeMultiplier is 1–18.
  • Although SensitivitySlider needs to be reversed to map to the underlying threshold value, VolumeSlider does not.

The code-behind for the settings page is shown in Listing 35.5.

LISTING 35.5 SettingsPage.xaml.cs—The Settings Code-Behind for Talking Parrot

[code]

using System.Windows;
using Microsoft.Phone.Controls;
namespace WindowsPhoneApp
{
public partial class SettingsPage : PhoneApplicationPage
{
public SettingsPage()
{
InitializeComponent();
}
// Simple property bound to the first slider
public int Volume
{
get { return Settings.VolumeMultiplier.Value; }
set { Settings.VolumeMultiplier.Value = value; }
}
// Simple property bound to the second slider
public int Threshold
{
get { return Settings.VolumeThreshold.Value; }
set { Settings.VolumeThreshold.Value = value; }
}
private void ResetButton_Click(object sender, RoutedEventArgs e)
{
this.VolumeSlider.Value = Settings.VolumeMultiplier.DefaultValue;
this.SensitivitySlider.Value = Settings.VolumeThreshold.DefaultValue;
}
}
}

[/code]

The Finished Product

Talking Parrot (Recording & Playing)

Cowbell (Sound Effects)

Cowbell is a simple musical instrument app. With it, you can tap the screen in any rhythm, and the app makes a cowbell noise with each tap. You can even play along with songs from your music library by switching to the Music + Videos hub, starting a song or playlist, and then switching back to Cowbell. The important aspect of Cowbell is that its sole purpose is to play sound effects.

Of all the musical instruments out there, why choose a cowbell? Many people find the idea of playing a cowbell entertaining thanks to a Saturday Night Live skit in 2000 with Will Ferrell and Christopher Walken. In it, Christopher Walken repeatedly asks for “more cowbell” while Will Ferrell plays it to Blue Öyster Cult’s “(Don’t Fear) The Reaper.” With this song in your music library and this app on your phone, you can re-create the famous skit!

Playing Sound Effects

On Windows Phone, Silverlight has only one way to play audio and video: the MediaElement element. However, this element is too heavyweight for playing sound effects. When it plays, it stops any other media playback on the phone (e.g. music playing in the background from the Music + Videos hub).

The relevant XNA class is called SoundEffect, and it lives in the Microsoft.Xna.Framework.Audio namespace. To use it, you must add a reference to the Microsoft.Xna.Framework assembly in your project. In this chapter, you’ll see how to load a sound effect from an audio file and how to play it.

Using MediaElement for sound effects could cause your app to fail marketplace certification!

Because using MediaElement for sound effects results in the poor user experience of halting background media,Microsoft checks for this when certifying your app for the marketplace. If you use MediaElement for sound effects, your app will not be approved for publishing.

If you need sound effects for your app and are unable to make them yourself, here are a few good resources to check out:

  • The Freesound Project (freesound.org)
  • Partners in Rhyme (partnersinrhyme.com)
  • Soungle (soungle.com)
  • Sounddogs (sounddogs.com)
  • SoundLab, a pack of game-centric sounds from Microsoft (create.msdn.com/ en-US/education/catalog/utility/soundlab)

The User Interface

Cowbell has a main page, an instructions page, and an about page. The latter two pages aren’t interesting and therefore aren’t shown in this chapter, but Listing 30.1 contains the XAML for the main page.

LISTING 30.1 MainPage.xaml—The User Interface for Cowbell’s Main Page

[code]

<phone:PhoneApplicationPage
x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
SupportedOrientations=”Portrait”>
<!– The application bar, with one button and one menu item –>
<phone:PhoneApplicationPage.ApplicationBar>
<shell:ApplicationBar Opacity=”.5”>
<shell:ApplicationBarIconButton Text=”instructions”
IconUri=”/Shared/Images/appbar.instructions.png”
Click=”InstructionsButton_Click”/>
<shell:ApplicationBar.MenuItems>
<shell:ApplicationBarMenuItem Text=”about” Click=”AboutMenuItem_Click”/>
</shell:ApplicationBar.MenuItems>
</shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>
<!– Just an image in a grid –>
<Grid Background=”Black” MouseLeftButtonDown=”Grid_MouseLeftButtonDown”>
<Image Source=”Images/cowbell.png” Stretch=”None”/>
</Grid>
</phone:PhoneApplicationPage>

[/code]

This is a simple page with an application bar and a grid with a cowbell image that handles taps with its MouseLeftButtonDown handler. For the sake of the cowbell image that has white edges, the grid is given a hard-coded black background. Therefore, this page looks the same under both themes except for the half-opaque application bar, as seen in Figure 30.1.

FIGURE 30.1 The main page looks identical on both dark and light themes, except for the application bar.
FIGURE 30.1 The main page looks identical on both dark and light themes, except for the application bar.

The Code-Behind

Listing 30.2 contains the code-behind for the main page. This is where all the soundeffect logic resides.

LISTING 30.2 MainPage.xaml.cs—The Code-Behind for Cowbell’s Main Page

[code]

using System;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Navigation;
using System.Windows.Resources;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Shell;
using Microsoft.Xna.Framework.Audio; // For SoundEffect
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
SoundEffect cowbell;
public MainPage()
{
InitializeComponent();
// Load the sound file
StreamResourceInfo info = Application.GetResourceStream(
new Uri(“Audio/cowbell.wav”, UriKind.Relative));
// Create an XNA sound effect from the stream
cowbell = SoundEffect.FromStream(info.Stream);
// Subscribe to a per-frame callback
CompositionTarget.Rendering += CompositionTarget_Rendering;
// Required for XNA sound effects to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
// Don’t let the screen auto-lock in the middle of a musical performance!
PhoneApplicationService.Current.UserIdleDetectionMode =
IdleDetectionMode.Disabled;
}
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
base.OnNavigatedFrom(e);
// Restore the ability for the screen to auto-lock when on other pages
PhoneApplicationService.Current.UserIdleDetectionMode =
IdleDetectionMode.Enabled;
}
void Grid_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
// The screen was tapped, so play the sound
cowbell.Play();
}
void CompositionTarget_Rendering(object sender, EventArgs e)
{
// Required for XNA sound effects to work.
// Call this every frame.
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
}
// Application bar handlers
void InstructionsButton_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/InstructionsPage.xaml”,
UriKind.Relative));
}
void AboutMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/AboutPage.xaml”,
UriKind.Relative));
}
}
}

[/code]

  • In the constructor, the stream for this app’s .wav audio file is obtained with the static Application.GetResourceStream method.
  • The use of the CompositionTarget.Rendering event is required for sound effects to work properly. This is explained in the following warning sidebar.

When playing sound effects with XNA, you must continually call Update on XNA’s framework dispatcher!

XNA’s sound effect functionality, like some other functionality in XNA, only works if you frequently (as in several times a second) call the static FrameworkDispatcher.Update method from the Microsoft.Xna.Framework namespace.This is natural to do from XNA apps, because they are designed around a game loop that runs code every frame. (XNA even provides a base Game class that automatically does this, so developers don’t have to.) From Silverlight apps, however, which are inherently event-based, you must go out of your way to run code on a regular schedule.

To call FrameworkDispatcher.Update regularly, you could use a DispatcherTimer, as done in previous chapters.You could even use a plain System.Threading.Timer because FrameworkDispatcher.Update can be called from any thread.

However,my preferred approach is to use an event Silverlight raises before every single frame is rendered.The event is called Rendering, and it is exposed on a static class called CompositionTarget.This event is useful for doing custom animations that can’t easily be represented with Silverlight’s animation classes from Part II,“Transforms & Animations,” of this book, such as physics-based movement. In Cowbell, the event is perfect for calling FrameworkDispatcher.Update with the roughly the same frequency that an XNA app would call it. Note that the first call to FrameworkDispatcher.Update is in the page’s constructor because it takes a bit of time for the first Rendering event to be raised.

If you call Play without previously calling FrameworkDispatcher.Update within a short time span, an InvalidOperationException is thrown with the following helpful message: FrameworkDispatcher.Update has not been called. Regular FrameworkDispatcher. Update calls are necessary for fire and forget sound effects and framework events to function correctly. See http://go.microsoft.com/fwlink/?LinkId=193853 for details.

  • The code in OnNavigatedTo and OnNavigatedFrom exists to ensure that the screen doesn’t auto-lock. If the cowbell player has a long break during a performance, it would be very annoying if the screen automatically locked. And tapping the screen to keep it active isn’t a good option, because that would make an unwanted cowbell noise!
  • The sound effect is played with a simple call to SoundEffect.Play in Grid_MouseLeftButtonDown. If Play is called before the sound effect finishes playing from a previous call, the sounds overlap.

The Audio Transport Controls

When the phone’s media player plays music, this music continues playing while apps run.Users can pause, rewind, fast forward, or change songs via the 93-pixel tall top overlay that appears on top of any app when the hardware volume buttons are pressed.This functionality works great with a fun instrument app such as Cowbell. In the next release of the Windows Phone OS, due by the end of 2011, third-party apps will be able to play music in the background just like the builtin media player.

The Finished Product

Cowbell (Sound Effects)