Future Topics

With so many new, exciting things coming around the bend it's difficult to determine where to start my new tutorial series(es). So, I figure the best way to write about what you want to know, is to ask what you want to know. Which of the following topics would you be most interested in learning about?

• .NET and CLR internals
• Migration from C# to C++
• Using C++/CX
• The WinRT API & Windows 8 Apps (Language agnostic)
• DirectX 11 via C++ (specify Apps or Desktop)
• DirectX 11 via C# (specify Apps or Desktop)
• Windows Phone 7 App/Game Development
• Windows Phone 8 App/Game Development
• General Game Development topics
• XNA 4.0 & PC/Xbox360/WP7 game development
• Visual Studio 2012 Tools, Debugging, & Best Practices
• WinDBG and low-level debugging of managed & native code
• Low-Level ASM programming and operating system concepts

Service Interruption

I will be bringing GameDevelopedia.com down within the next 24-48 hours for a hardware & software upgrade. This is to ensure it's running with the most up-to-date and secure software, and to ensure it scales well with the plans I have in store for GameDevelopedia.com and the server it runs on.

Return from the Hero's Journey

So... It's been a while since I posted. In fact, it's been so long that when I decided I had enough free time and interest to start posting again, I almost decided to wait the three weeks just so I could post an anniversary edition. But alas, I figured the best thing to do was to just make the first post and get it over with.

So, what's been up with me, and why the h3ll have I been gone so long? Well, first and foremost (and most should already know this), I now work at Microsoft. Actually, I've been working at Microsoft for nearly a year now, which not coincidentally, is one of the main reasons I haven't posted in nearly a year. The last year has been a fairly intense roller-coaster ride of learning, facing new challenges, and taking on new responsibility. Shortly after joining Microsoft I decided I wanted to be among the elite here, and there were still skills I hadn't mastered and technology I was unfamilir with. So over the last year I've made it a personal goal to have no fear, to face my long-time demons and to come out wiser for it.

As a result, in the last year I've challenged myself by taking courses, reading about, or playing with:

• x86 & amd64 Assembly language
• Debugging with WinDbg (Native & SOS/SOSEX for Managed Code)
• Deep-diving into the .NET CLR to gain deeper insight into generational garbage collection & the managed API
• Multi-Threaded programming
• TCP/IP Socket Programming
• .NET technologies such as Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF)
• Windows Internals & SysInternals

XNA 4.0 Workshop - Week 5

Week five is the first week of the XNA 4.0 workshop where I'll step away from doing an overview of the "textbook" chapters, and instead focus on the weekly topics themselves.

In Week five we take a look at XNA 4.0's Audio/Video features. If you DO want to read the chapters in the "textbook", check out chapters 7 and 12. But in truth, you'll find everything you need to know below, or in the external resources provided at the bottom.

XNA's Audio/Video support can be broken up in to a few smaller categories:

1. Playing 2D Audio
2. Playing 3D Audio
3. Accessing Media
4. Recording Audio
5. Playing Video

Playing 2D Audio

The first of the four categories above "Playing 2D Audio" can itself be broken up into two categories:

• Custom Audio Playback
• Simple Audio Playback

Custom Audio Playback

Prior to XNA 3.0 the only method for Audio playback was the Custom Audio Playback method. This method entails using the XACT Framework (Cross-Platform Audio Creation Tool) to combine wave files into sounds and then bind sounds to cues to be triggered at run-time. The benefit to the XACT method of creating sounds is it cleanly separates the job of audio designer/composer from programmer. Engineers are freed from needing to know what sounds to play and when. All they need to do is use the in-game API to trigger cues, and whatever nifty effects have been created by the audio engineer will play. Additionally, using the XACT method sounds can be cleanly grouped into categories, with volume control and other settings being configurable at a categorical level. This is great for allowing users to modify the ambient sound volume, or music volume, while leaving the sound effect volume alone.

The main downside to the XACT method is it doesn't go through the traditional content pipeline XNA developers are used to. The XACT tool instead creates project files which reference external wave files, and then these XACT project files are added as a resource to the XNA Content Pipeline. This necessarily changes the way sounds are visualized in the Content folder, and also how they're accessed in game. To access the sound cues, programmers are required to instantiate bank objects, and then make regular calls to the AudioEngine class which is responsible for processing audio data.

The main classes involved in the Custom Audio Playback method are:

• AudioCategory
• AudioEngine
• Cue
• SoundBank
• WaveBank

All of the above classes can be found in the Microsoft.Xna.Framework.Audio namespace.

Simple Audio Playback

Beginning with XNA 3.0 Microsoft introduced the SoundEffect class. SoundEffect is an in-game asset type similar to Texture, SpriteFont, etc... which maps one-to-one with an on-disk sound file. SoundEffect objects can be associated with sound files of different types, including WAV, WMA, or MP3.

Unlike with the XACT Framework, there's no good way to create randomization over which sound will play when you call SoundEffect.Play, there's no way to make it cross-fade between multiple sounds, and there's no good way to make multiple waves play at the same time, or with time delays. This level of customization requires the use of the XACT Framework.

What you do get with the SoundEffect class is the ability load an audio file with one line of code, and play it with another. It's quick, easy, and follows the usual method of getting assets into the game.

In addition to just being able to "fire and forget" sound effects using the SoundEffect class, engineers can also get access to a SoundEffectInstance object of an associated SoundEffect, and then pause, play, or resume playback while being able to adjust things like pan, pitch, and volume.

For 90% of games out there the Sound Effect API will work just fine. Also note that the XACT Framework isn't available for Windows Phone 7. So if your plan is to hit the Windows Phone Marketplace, SoundEffect and its instance class are your only options.

In addition to the SoundEffectInstance class which was introduced in XNA 3.0, Microsoft introduced the DynamicSoundEffectInstance class in XNA 4.0. This class allows you to work directly with audio buffers to enable the dynamic creation of audio at run-time, and can also be used to implement streaming of audio from disk.

The main classes involved in the Simple Audio Playback method are:

• DynamicSoundEffectInstance
• SoundEffect
• SoundEffectInstance

All of the above classes can be found in the Microsoft.Xna.Framework.Audio namespace.

Playing 3D Audio

Whether you're using custom sounds or the simple sound effect API, there will come a time when you may want to work with 3D (positional) sound. Positional sound works the same as normal audio, except that the volume will rise/lower and interpolate between speakers based on the origin of a sound relative to the listener. This allows you to easily create effects where audio sounds as though it's coming from the left or right of you. I have never tried using the 3D Sound API with more than two speakers, so I'm not sure how well it works with 5.1 and 7.1 speaker setups.

To use 3D audio with XACT, simply call Cue.Apply3D, passing in an emitter and listener. Similarly, to use 3D audio with the SoundEffect API, call SoundEffectInstance.Apply3D.

The main classes involved in 3D Audio are:

• AudioEmitter (used to set data for the origin)
• AudioListener (used to set data for the listener)

All of the above classes can be found in the Microsoft.Xna.Framework.Audio namespace.

Accessing Media

Up until this point I've mostly been assuming engineers will be accessing their audio through the Content Pipeline, using either XACT projects or sound files which are loaded at run-time and then played. However, beginning with XNA 3.0, when Microsoft added (and subsequently removed) support for the Zune, they also added access to your system's media library through the MediaLibrary class. This includes all the songs, albums, cover art, and play lists in your audio collection, and also the pictures and videos (more on this later) in your Libraries.

These assets are all accessed using intuitively named classes in the Microsoft.Xna.Framework.Media namespace in combination with the MediaPlayer class, another addition in XNA 3.0.

To use the MediaPlayer class just call one of the static functions such as MediaPlayer.Play and pass in the Song you want. You can filter the song you want by using the MediaLibrary.Songs collection, or if you want more control you can first access the albums in your library through the MediaLibrary.Albums property, etc...

With XNA 4.0 also comes a nifty new method on the Song class called Song.FromUri. This method allows you to play a song from the web or from file, but does not work on the Xbox 360. It also doesn't support spaces in the path name, so make sure that doesn't happen or you'll get an exception.

The main classes involved in accessing audio from the media library are:

• Album, AlbumCollection
• Artist, ArtistCollection
• Genre, GenreCollection
• MediaLibrary
• MediaPlayer
• MediaQueue
• PlayList, PlayListCollection
• Song, SongCollection

Recording Audio

Being able to play audio is all well and good, but we also need to be able to record or stream microphone audio. Beginning with XNA 4.0 Microsoft has added the Microphone class to the Microsoft.Xna.Framework.Audio namespace.

Using the Microphone.Default or Microphone.All properties developers can gain access to one or more Microphone objects hooked up to their Phone, PC, or Xbox 360. Once you have a reference to the Microphone, it's as easy as calling Microphone.Start and Microphone.Stop in order to record audio. Then, using the Microphone.GetData method (in response to BufferReady events) developers can gather the audio as a byte[] array to use in combination with the DynamicSoundEffectInstance class discussed previously to play back the audio.

There are of course other properties on the Microphone class to help you identify important information such as the SampleRate, and BufferDuration, and to determine whether it's a headset or not, but you get the basic idea. Play, queue buffers, stop.

As an aside I should note that recording audio is meant to record audio for use in a game. If you just want to use a microphone to communicate with your friends over Xbox Live, that comes for free.

If you're taking advantage of the GamerServices API you can call LocalNetworkGamer.EnableSendVoice to allow audio to be transmitted. With the addition of several read-only properties such as NetworkGamer.HasVoice, NetworkGamer.IsTalking, and NetworkGamer.IsMutedByLocalUser, developers can add interface elements to indicate who can/is talking and whether they're muted.

Playing Video

And finally we come to playing video. Starting with XNA 3.1 Microsoft introduced the VideoPlayer class, located in Microsoft.XNa.Framework.Media. This class allows you to play videos (obviously).

To use it you simply:

• Construct an instance of the VideoPlayer class
• Call VideoPlayer.Play(video)

Then each frame call VideoPlayer.GetTexture and it will return, sequentially, the frames of the video as a Texture2D object. This is great because it means it can be used with SpriteBatch to draw your video as an animated sprite, or applied to a 3D quad to be rotated, stretched, and then projected into any 3D universe. Great for adding video to signs, on soda vending machines, etc...

Initially the only way to gain access to a Video was through the Media Library (discussed previously), but in XNA 4.0 Microsoft added Video as an asset type, allowing you to Content.Load any WMV file via the Content Pipeline.

External References

MSDN Documentation: 

• Creating and Playing Sounds
• Displaying Pictures and Playing Video 

AppHub Education Catalog

• Music Management for Windows Phone 7
• Silverlight Sound Sample
• Silverlight Microphone Sample
• Microphone Echo
• Sound & Music
• Dynamic Audio
• SoundLab
• 3D Audio

XNA 4.0 Workshop Textbook Re-Evaluation

When the XNA 4.0 Workshop first began back in January I did a survey of the available XNA books. At the time, there were very few XNA 4.0 books available, and even among the 3.0 books there weren't many that I felt covered the topics in sufficient depth and in the correct order. After careful consideration I ultimately chose XNA Game Studio 3.0 Unleashed as the "textbook" for this workshop.

Unfortunately, the book covers XNA 3.0, not 4.0 which has made it difficult for people following the workshop to get projects functioning, as there were several breaking changes between XNA 3.1. Additionally, after eight months, the AppHub has come a long way in providing additional resources for those wanting to learn XNA 4.0.

So my plan moving forward is to, in the weekly workshop introduction, identify the topics we'll be covering for the week, point to a few resources on the AppHub which may help, identify some documentation pages in the MSDN Library, and then wherever necessary write additional tutorials to supplement the information already available. Also, I will indicate which chapters in the textbook we'll be looking at for those who purchased the book and would like to follow along that way. 

What this ultimately means is it doesn't matter which XNA books you have, or if you have none at all. Hopefully by addressing the topics, documentation, and tutorials already available I can make the workshop more accessible to everyone.

SpriteBatch Then and Now

This week in the XNA 4.0 workshop we're covering 2D graphics using the SpriteBatch class. For those that used XNA 3.1 or earlier, and especially those that have never used DirectX, the new arguments to SpriteBatch.Begin can be a bit off-putting. 

When using the SpriteBatch class there are really just four functions that'll you'll use most of the time. These are

• Draw()
• DrawString()
• Begin()
• End()

The first two are fairly obvious. Any time you want to draw a sprite or a string you call the corresponding function. The latter two need some explanation.

XNA is built on DirectX which is a hardware accelerated API for manipulating 3D primitives. This means that everything we do with XNA is really in 3D using triangles. Drawing sprites is no exception. Every time you call Draw or DrawString it's really just queuing a texture to be drawn on a 3D quad rendered in pre-transformed screen space.

Because XNA/DirectX is really 3D, the API has to do some initial setup with the Graphics Pipeline to make sure things are drawn with the appearance of 2D. In XNA 3.1 and prior this was done with shader code that was called from SpriteBatch during the drawing process. When that happened depended on which sort mode you were using. But ultimately your textures, combined with specific device states, would be drawn to the screen in a series of pre-transformed quads.

In XNA 3.1 you had a little bit of control over this process, but not much. When you called Begin you could pass it a number of arguments depending on which overload you chose. The most complicated overload looked like this:

Begin(SpriteBlendMode, SpriteSortMode, SaveStateMode, Matrix)

In the above overload, SpriteBlendMode, SpriteSortMode, and SaveStateMode were all just simple enumerations.

SpriteBlendMode was used to determine how sprites you drew were combined with data already in the target buffer. The options were either SpriteBlendMode.Additive, SpriteBlendMode.AlphaBlend, or SpriteBlendMode.None.

SaveStateMode was used to determine whether or not device states would be reset after calling Begin/End on the SpriteBatch class. The options were SaveStateMode.None and SaveStateMode.SaveState. The default was None, and would often cause confusion for people when switching back and forth between drawing with SpriteBatch and without in the same draw pass. Suddenly, people's 3D models would draw without respect for the depth buffer and all of their alpha blending states would be reset.

SpriteSortMode was used to determine in what order your sprites were drawn. The possible values were: BackToFront, Deferred, FrontToBack, Immediate, Texture.

In deferred mode all of your draw calls were deferred, and nothing was actually flushed to the hardware until End was called. Once End was called, the device had all of its render states set and all of your sprites were drawn to the target surface in the same order as they were "drawn". Of course, if you drew your sprites in something other than a painter's algorithm, or if you didn't intelligently split up your sprites by texture, it could result in undesirable layering or poorer performance, especially if you were rendering sprite-based particles.

The BackToFront and FrontToBack sort modes were designed to handle the painter's algorithm. That is, by passing a z-value in with your Draw calls you had more control over the order in which sprites were drawn. In these modes, it first sorted all your sprites by the z-value, ascending or descending, and then drew them. So it didn't matter the order in which you called Draw.

The Texture mode is/was for performance. If you drew from several different texture sheets within the same Begin/End pair, each switch from one texture to another would cause a flush and the device had to make a unique DrawPrimitive DirectX call. This is bad m'kay. By using the Texture mode it first sorted all of your sprites by texture, and then called them with as few DrawPrimitive calls as possible. Note, if all your sprites are/were from the same (or relatively few) texture sheet(s), then Texture sorting is useless, and you're better off using BackToFront or just Deferred.

The Immediate mode is interesting. With Immediate sort mode the device state was initialized as soon as you called Begin, and then each time you called Draw the quad was immediately pushed to the target surface. This is bad performance, but gave you control over your sprite rendering in a way none of the previous sort modes did. This is because you could call Begin, then manually make changes to the GraphicsDevice, such as changing blending values, filtering modes, depth buffer flags, etc... and THEN draw your quads. The problem was, now all of those quads were being drawn individually.

Enter XNA 4.0. With XNA 4.0, Microsoft acknowledged a need to give greater control over sprite drawing, while still maintaining performance. So the SpriteBatch.Begin method has a different set of arguments. The corresponding overload now looks like this:

Begin(SpriteSortMode, BlendState, SamplerState, DepthStencilState,
    RasterizerState, Effect, Matrix)

Notice that gone are most of the enumerations. While there is still an enumeration for SpriteSortMode, BlendState, SamplerState, DepthStencilState, RasterizerState, and even Effect are all fully realized objects. This allows you to set properties on the device at the time you call Begin, and then have those states be set for the duration of the Begin/End pair. So you can set certain properties, set your Sort mode to Defferred, and gain the benefit of both custom settings and batched rendering.

If you're like most people, when you see that set of object parameters your first instinct is to go..."uh, how do I set all those." Fortunately for us, Microsoft provided static fields on each of those classes which grants access to instances of the most common scenarios you'll need when rendering with SpriteBatch.

• For BlendState, there are static fields for Additive, AlphaBlend, NonPremultiplied, and Opaque Blend States.
• For SamplerState there are several static fields, but you'll most often use the default, which is LinearClamp.
• For DepthStencilState there are Default, DepthRead, and None. Interestingly enough, Default is pre-configured with the default states you'll want set if you want to use the stencil buffer in your 2D rendering. In truth, None is actually the "default" setting.
• And last, RasterizerState includes CullClockwise, CullCounterClockwise, and CullNone. The default is CullCounterClockwise.

You'll also notice from the above prototype the addition of the Effect parameter. This allows you to pass in your own custom effect, so you could even do your 2D drawing in 3D if you wanted to. It helps to have something to start from though if you're going to provide your own custom effect. That's why Microsoft has released the Source Code to the SpriteEffect, which can be downloaded from the AppHub.

If at the end of the day you just want to draw in 2D and don't need any of the options you can just call the simpler version of the Begin method, or use the above but pass in null for as many arguments as you like. Since those are objects and not enumerations, it's fine to just let the SpriteBatch class determine what the most appropriate paramter is.

So you see, with the transition from XNA 3.1 to XNA 4.0 Microsoft has given users a lot more control over how they do 2D rendering with the SpriteBatch class. By messing with the different state objects you can make your sprite blend with the surface in new and unique ways, have it draw to the stencil buffer, ignore (or not) the depth buffer, or even have sprites wrap within the target rectangle. Because you can also provide your own effect it's now also possible to do things such as having your sprites cast shadows, or react in unique ways to custom lighting, all without sacrificing performance.

Right Brain Re-Loaded

When I was younger I had an interest in drawing. I also had an interest in video games and programming. For better or worse, I ended up going the latter route but several times over the course of my life my right hemisphere has demanded to be heard. Short bursts of creativity will come out where I'll be forced to learn a new instrument (technically left brained), write a story, or do some artwork. (I think it's the Gemini in me)

I'm once again going through a right brained period of my life. I've found myself buying books on drawing, re-installing Photoshop and Blender, and carrying a sketch book around with me. The other day I decided to sit down and see if there was still any talent left up there, or whether my right brain had entirely shriveled. Here was the result. What do you think Internet, is there hope for me to become that rare combination of programmer and artist?

Giving Blender a Second Chance

One of the biggest obstacles I face as a professional game programmer is finding artwork for my games. This is true whether we're talking about concept art, textures, or even 3D environment art or characters. While there are tons of game designs out there that don't require good looking artwork, I'm just not that creative. My games all require fantastic artwork, otherwise they likely wouldn't be interesting enough to merit purchase or even play, really. 

As a result, I have, over the years, experimented with Photoshop, Maya, Max, and even XSI. The problem with most of these tools is they're expensive. Granted, most have free/learning editions, but if I want to do any exporting or writing plug-ins I need a fully functional version.

Enter Blender. Blender has for a number of years been the defacto 3D modeling package for people on a budget. That is to say, it's the least expensive of all the modeling packages (free is always in my price range). Unfortunately, Blender has also become notorious for having a cumbersome, difficult to use interface which only Jedi Masters can learn to use. Once learned, however, evidence suggests a dedicated artist can create amazing things with Blender.

Even now I don't have the free time necessary learn to use the complicated Blender interface. Which is why I was pleasantly surprised when out of sheer necessity I downloaded the latest version of Blender and discovered it looks like the image below! Alas, it looks more like a modern 3D modeling package. It shares striking resemblance to 3DS Max and XSI now. I guess it's time for me to Give Blender a second chance and see if I can't finally get some decent artwork for my otherwise un-interesting or innovative games.

XNA 4.0 Workshop - Week 4

Supplemental Reading

XNA 4.0 Source Code

Discussion

XNA 4.0 Workshop - Refresh

Back in January I began the XNA 4.0 Winter Workshop with the goal of aiding people in their journey to learn XNA. The goal was for it to be a guided workshop, where each student takes responsibility for reading the material introduced each week and learning the information, with me (or other tutors) identifying the topics to be learned and where to learn that information - starting with the chosen textbook. Because everyone was to be working from the same source material, students should have found it easier to get answers to the specific questions they had.

Unfortunately, real-life issues, both personal and professional made it impossible for me to uphold my end of the bargain. While I had provided an outline of what topics to cover each week, I wasn't able to post study notes, resource lists, etc... and so after about the third week participation ended for most people.

Summer is here. While I am busy writing curriculum for the high school students at DigiPen, I'm not in class every day so my schedule is considerably more flexible. With that in mind, I'd like to give the workshop another go and make myself available to all those out there who want to either pick up where we left off, or use this as an opportunity to start learning XNA 4.0 from the beginning.

Before we move on with week 4, I want to quickly provide an overview of the topics we've covered so far and provide rationale for putting them in that order.

Week 1 - The first week of the XNA 4.0 Workshop was focused on learning exactly what XNA 4.0 is, as well as how to get it installed on your computer. Week 1's reading material was chapters 1-3 in our textbook, "Microsoft XAN Game Studio 3.0". As we're using XNA 4.0, there were some small changes that people needed to be made aware of. In specific, I posted an article on time stepping and the Main Game Loop, which was part of the information covered in the text. You can find the complete Week 1 post here: XNA 4.0 Workshop - Week 1

Week 2 - The second week of the workshop jumped right into 3D mathematics and computer graphics. This seems a bit extreme for most people, but the reason I did this is because while 2D game development is mathematicslly simpler, all modern games are inherently 3D, even the ones that look 2D, because they're being developed with a 3D API on 3D hardware. I wanted the readers to understand that when we finally got to the chapters on 2D graphics, primarily for performance reasons; as well as simply pulling back the curtains.

I think it was in this Week that I started to lose people. The reason for this is that for many, the hardest part of working with 3D graphics is the mathematics required; specifically trigonometry & linear algebra. There is a very specific process for taking 3D geometry in model space, and converting it into a 2D image in screen space. This process involves a set of tranformations most often done with matrices, and applied to the geometry's primitives - in this case vertices (points with extra data). It is impracticle for me to try and provide a single tutorial or outline on how to perform the math for this process. But, there are a number of great resources out there if you do a Google Search. The goal of Week 2 is for you to get an understanding of the 3D mathematics required, and to bring your questions to our official workshop forums for answers. I'll continue to collect a list of resources for Week 2 and put them on the XNA 4.0 Workshop - Week 2 post.

Week 3 - The third week of the workshop focused on a few different topics related to user input. In specific, it addressed the different input devices available for use with XNA 4.0, as well as how to create a simple wrapper class and then make a service out of it. The weekly reading ended with creating a camera and then a split screen which demonstrated moving around in a world. If you got lost in Week 2, Week 3 was only going to be harder. For a look back, see the XNA 4.0 Workshop - Week 3 post.

And that brings us up to where we are now. In week 4, we're going to take a step back from 3D and focus on 2D. This will be easier for many people, as the classes are more straight forward and there's no complicated matrix mathematics that needs to be done. After reading over the material for Week 4 you may find it beneficial to go back and read over weeks 2 and 3 again. Also, I'm going to make an attempt at providing many more, smaller tutorials related to the weekly topics. That way I can push them out quicker, and you all can see the simplest cases possible when trying to understand the topics.

For those that would like a reminder as to what topics we'll be covering in the XNA 4.0 Workshop, take a look at the XNA 4.0 Workshop Schedule & Topics.

If you're just now joining us or are returning to the workshop I say Welcome! I'll provide the weekly post for Week 4 as soon as I can.