Saturday, November 26, 2011

On "Book Code"

Ok,
So you buy that nifty new book on Game Programming. One would assume the people writing the chapters would know what they are doing, or at least have used the code in an actual game.  Nope.

Always understand what the code does, and be ready for problems arising from the difference between your goal, to write a working commercial game, and the writer's goal, to write a chapter in a book.

I have run into this in two places recently.  One is the network library MTUDP in

Advanced 3D game programming with DirectX 10.0 and the other has been bullet physics. 

MTUDP is a good idea but not done and tested well. (If you want my rewrite, email me) and bullet physics is a university project to try new ideas in physics.  In my case I want to create a open ended infinite zombie game with on-the-fly AI generation of all game content.

There is an echo here of operating system issues.  Was the OS written to make money for a huge corporation, or by hobbyists to make a system they want to work in and use, or as a fashion statement for elite computer users? (Ok, you guess which three OSs to which I am referring.0

Oh, well, end of rant. And end of three weeks of wasted time trying to debug and make a fundamentally broken library work.

TF

Monday, November 21, 2011

MTUDP.cpp More Fixes

WARNING - I should remove this post, but instead will give a warning.  Do not use the MTUDP.cpp library as is.  It has many bugs and makes a fatal mistake.  The user ID is the IP address.  This breaks if there is a fire wall in the way, and there is always a firewall.  I have nearly rewritten the library with very little of the original code left over.

--------------------------------------------------------------------

Ok, turns out the previous post was not a complete fix for the Ack code.
As usual the blog mangles the C code a bit, but you should be able to copy this, else email me for a complete copy.



 unsigned short NetLibHost::ProcessIncomingACKs( char *pBuffer, unsigned short len, DWORD receiveTime )
 {
  UNREFERENCED_PARAMETER(len);
  // Get the number of ACKs in this message, not counting the base ACK.
  unsigned char numAcks, mask, *ptr;
  DWORD         basePacketID, ackID;

  ptr = (unsigned char *)pBuffer;

  // The story: We want to ack each received packet. 
  // But if we have received a series of packets we only
  // have to ack the highest numbered one and we assume all the lesser packets are ack'd too.
  // But then there can be some higher number packets we have received with some gaps.
  // So we send the highest received packet so far with no unreceived packets less than it, then
  // have bytes with the bits representing yes/no acks for higher numbered packets.'
  // So say we have received  3,4,5,6,7,9,10,11,13
  // The base would be 7 since we have all pacjets up to that,
  // Then the next byte is the bits 0x00 | 0x40 | 0x20 | 0x10 | 0x00 | 0x40
  // for packets 8,9,10,11,12,13 respectively.

  // Get the base packet ID, which indicates all the ordered packets received so far.
  memcpy( &basePacketID, ptr, sizeof( DWORD ) );
  ptr += sizeof( DWORD );
  // Get the number of additional ACKs.
  // TODO - Keene - Runs off the end if Ack record was truncated to fit!
  // Solution: don't ever make packets messages that are too large, e.g. 3k
  numAcks = *ptr;
  ptr++;
  // Zero the byte so if there is a one off error in bits, it is not a false ack.
  *ptr = 0x00;
  ackID = d_outQueue.GetLowestID();

#if defined( _DEBUG_VERBOSE )
  OUTPUTREPORT3( "<   Ack low=%04d base=%04d end=%04d\n", ackID, basePacketID, basePacketID + numAcks );
#endif

  // Can get stuck in loop here if corrupt data.
  int debugCount = 0;
  while( ackID <= basePacketID )
  {
   debugCount++;
   // The packet has been ack's so update average ping time.
   ACKPacket( ackID, receiveTime );
   ackID++;
  }

  mask = 0x80;

  // TODO - Keene - Runs off the end if ack record was truncated to fit!
  // Solution: don't ever make packets messages that are too large, e.g. 3k
  while( ackID < basePacketID + numAcks )
  {
   if( mask == 0x00 )
   {
    mask = 0x80;
    ptr++;
    // Zero the byte so if there is a one off error in bits, it is not a false ack.
    *ptr = 0x00;
   }

   if( ( *ptr & mask ) != 0 )
   {
    ACKPacket( ackID, receiveTime );
   }

   mask >>= 1;
   ackID++;
  }

  return (unsigned short)(ptr - (unsigned char *)pBuffer);
 }

 unsigned short NetLibHost::ProcessIncomingReliable( char *pBuffer, unsigned short maxLen, DWORD receiveTime )
 {
  maxLen;
  // Process any messages in the packet.
  DWORD           packetID;
  char            *readPtr;
  unsigned short  length;

  readPtr = pBuffer;
  memcpy( &packetID, readPtr, sizeof( DWORD ) );
  readPtr += sizeof( DWORD );
  memcpy( &length, readPtr, sizeof( unsigned short ) );
  readPtr += sizeof( unsigned short );
#if defined( _DEBUG_VERBOSE )
  OUTPUTREPORT2( "<   %04d (%d) R\n", packetID, length );
#endif
  // If this message is a packet, queue the data
  // to be dealt with by the application later.
  d_inQueue.AddPacket( packetID, (char *)readPtr, length, receiveTime );
  readPtr += length;

  // Should we build an ACK message?
  if( d_inQueue.GetCount() == 0 )
  {
   return (unsigned short)( readPtr - pBuffer );
  }

  // Build the new ACK message.
  DWORD         lowest, highest, ackID;
  unsigned char mask, *ptr;

  lowest = d_inQueue.GetCurrentID();
  highest = d_inQueue.GetHighestID();

  // Cap the highest so as not to overflow the ACK buffer
  // (or spend too much time building ACK messages).
  // (Was bug here because ACK_MAXPERMSG was 256 which does not fit in a byte.)
  if( highest > lowest + ACK_MAXPERMSG )
  {
   highest = lowest + ACK_MAXPERMSG;
  }

#if defined( _DEBUG_VERBOSE )
  OUTPUTREPORT2( " >  %04d ack to %04d ", lowest, highest );
#endif

  // The story: We want to ack each received packet. 
  // But if we have received a series of packets we only
  // have to ack the highest numbered one and we assume all the lesser packets are ack'd too.
  // But then there can be some higher number packets we have received with some gaps.
  // So we send the highest received packet so far with no unreceived packets less than it, then
  // have bytes with the bits representing yes/no acks for higher numbered packets.'
  // So say we have received  3,4,5,6,7,9,10,11,13
  // The base would be 7 since we have all pacjets up to that,
  // Then the next byte is the bits 0x00 | 0x40 | 0x20 | 0x10 | 0x00 | 0x40
  // for packets 8,9,10,11,12,13 respectively.

  ptr = (unsigned char *)d_ackBuffer;
  // Send the base packet ID, which is the ID of the last ordered packet received.
  memcpy( ptr, &lowest, sizeof( DWORD ) );
  ptr += sizeof( DWORD );
  // Add the number of additional ACKs.
  *ptr = (unsigned char)(highest - lowest);
  ptr++;
  // Zero the byte so if there is a one off error in bits, it is not a false ack.
  *ptr = 0x00;

  ackID = lowest + 1;
  mask = 0x80;

  while( ackID <= highest )
  {
   if( mask == 0x00 )
   {
    mask = 0x80;
    ptr++;
    // Zero the byte so if there is a one off error in bits, it is not a false ack.
    *ptr = 0x00;
   }

   // Is there a packet with id 'ackID' ?
   if( d_inQueue.UnorderedPacketIsQueued( ackID ) == true )
   {
    *ptr |= mask;  // There is
   }
   else
   {
    *ptr &= ~mask;  // There isn't
   }

   mask >>= 1;
   ackID++;
  }

#if defined( _DEBUG_VERBOSE )
  OUTPUTREPORT0( "\n" );
#endif

  // Record the ammount of the ackBuffer used.
  d_ackLength = (unsigned short)(ptr - (unsigned char *)d_ackBuffer);
  assert(d_ackLength <= ACK_BUFFERLENGTH);

  // return the number of bytes read from buffer
  return (unsigned short)( readPtr - pBuffer );
 }


Sunday, November 20, 2011

MTUDP.cpp NetLib Bug fixes.

WARNING - I should remove this post, but instead will give a warning.  Do not use the MTUDP.cpp library as is.  It has many bugs and makes a fatal mistake.  The user ID is the IP address.  This breaks if there is a fire wall in the way, and there is always a firewall.  I have nearly rewritten the library with very little of the original code left over.

--------------------------------------------------------------------

I have been using the MTUDP.cpp library from the book
Advanced 3D game programming with DirectX 10.0 By Peter Walsh


The library has many bugs that I have been fixing over time.


Several are...

The library send all packets twice, which is unnecessary. So in MTUDP.cpp I comment out the three sections where that happens. Look for secondPacket to find the blocks.

In NetLibHost.cpp there is a big problem with the Ack generation loops. They initialize the mask to 0x80 and if there are no additional Acks, then the later count returned is off by one since the mask is not zero. The mask should be initialized to 0x00. (two places in the code. Search for mask = 0x80;) WARNING - WRONG - SEE NEXT POST

The Ack record can be truncated in AddAckMessage it checks to see if the the length is exceeded and if it is it truncates, but the count of additional acks is not adjusted, boom! I don't have a fix for this except make sure no packets approach the max packet size boundary. I put an assert in to check if you run off the end.

I noticed that on creation the Host record does not initialize the ping times and last packet time. Hmmm, haven't worked on that one yet.

Hope this saved some poor soul some time.



TF