Yr12 Journal 22

In Term 3 Week 9, I tested my project again and I was also doing a C# project.

Progress

Testing

As usual, I tested my project, it is definitely accurate again this time:

Therefore, this model is elegant and accurate, it makes reliable predictions.

My C# Project

As there is nothing much to test, and my model is already good enough, I decided to do write some C#.

Situation

So I have a tcp client and server. I need them to communicate. And there is a router that redirects the message between the client and the server. The client will connect to the router, and the router will redirect the binary data to the server. This structure can prevent DDOS and CC attack. The server application will be incredibly safe because no one knows its address and no one can send stuff directly to it.

However, suppose we have 1000 clients, and the server needs to broadcast message (like a chatroom) to those clients everytime when it receives a message. If each client send 1 message per second, the server will receive 1000 messages redirected from the router, and it will send the router 1000 * 1000 messages (to broadcast). That indicates the router needs to accept 1000 messages per second from the clients and it needs to accept 1,000,000 messages from the server then distribute them to each client.

If we do not merge the binary data transaction between the router and the server, there would be a IO leak, and a CPU leak when they have to send and accept too many small buffers at a short time. Hence, we need to merge the messages using packets and a buffer, then send them between server and router in constant interval.

Packet Constraints

The packet has the following info:

• Length of this part of message (ushort, 2 bytes, 65535 maximum)
• Session Id, indicates what client on the particular router owns this session (ushort, 2 bytes, 65535 maximum)
• Data, byte array

Buffer Design

We need a buffer that is thread safe, and is concurrent, and hopefully we can manage its allocation (so GC would not notice this buffer).

Hence, by using Nino.ExtensibleBuffer (made by myself), the following code was designed:

public class BufferPool
{
    /// <summary>
    /// pool
    /// </summary>
    private readonly ExtensibleBuffer<byte> buffer = new ExtensibleBuffer<byte>(1024 * 100);//100kb extensible size

    /// <summary>
    /// cur pos
    /// </summary>
    private int pos;

    /// <summary>
    /// whether or not has data
    /// </summary>
    public bool Valid => pos > 0;
    
    /// <summary>
    /// write single byte
    /// </summary>
    /// <param name="b"></param>
    public void Write(byte b)
    {
        lock (this)
        {
            buffer[pos++] = b;
        }
    }

    /// <summary>
    /// write byte array
    /// </summary>
    /// <param name="bytes"></param>
    public void Write(byte[] bytes)
    {
        lock (this)
        {
            buffer.CopyFrom(bytes, 0, pos, bytes.Length);
            pos += bytes.Length;
        }
    }

    
    /// <summary>
    /// byte array to send
    /// </summary>
    /// <return></return>
    public byte[] GetSendBuffer()
    {
        lock (this)
        {
            var ret = CompressMgr.Compress(buffer,pos);
            pos = 0;
            return ret;
        }
    }
}

public class PacketHelper
{
    public static int PacketLength = 2 + 1 + 1 + 2;

    /// <summary>
    /// write to specific buffer
    /// </summary>
    /// <param name="buffer"></param>
    /// <param name="serverType"></param>
    /// <param name="msgId"></param>
    /// <param name="sessionId"></param>
    /// <param name="data"></param>
    public static void WriteToBuffer(BufferPool buffer, Server.Type serverType, byte msgId, ushort sessionId,
        byte[] data)
    {
        ushort len = (ushort)(PacketLength + data.Length);
        //len
        buffer.Write((byte)len);
        buffer.Write((byte)(len >> 8));
        //sessionId
        buffer.Write((byte)sessionId);
        buffer.Write((byte)(sessionId >> 8));
        //write data
        buffer.Write(data);
    }

    /// <summary>
    /// read a packet
    /// </summary>
    /// <param name="buffer"></param>
    /// <param name="offset"></param>
    /// <returns></returns>
    public static (ushort packetLen, ushort sessionId, byte[] data) ReadPacket(ArraySegment<byte> buffer, int offset)
    {
        //read data
        var len = (ushort)(buffer[offset + 0] | buffer[offset + 1] << 8);
        var sessionId = (ushort)(buffer[offset + 2] | buffer[offset + 3] << 8);
        //request array from array pool
        byte[] data = ArrayPool<byte>.Request(len - PacketLength); //skip header len
        buffer.Slice(offset + PacketLength, data.Length).CopyTo(data);
        return (len, sessionId, data);
    }
}

This code can write data to extensible buffer, convert to compressed (zlib algorithm) byte array to send, and can read packet from ArraySegment with specific offset. One thing to be carefore is to decompress before reading packet.

Challenges

Writing C# code is so much more interesting than writing python, so obviously there were nothing challenging.

Reflection

This week I did no school work because I already finished my school IT project, but I was still learning stuffs in IT area. Therefore, overall I think I did well this week, and I used my class time wisely to think about these codes in my head. Moreover, I think I should use my school IT class time to write my ANU H Course project, as it is due next week Thursday. But overall I think this week was fine.

Timeline

I followed up my schedule, as I successfully tested my project and it still works. Next week would be my second last week testing it, then I will start making my documentation and presentation.