Handles, Eventers and Events

This guide is mostly ment for people who want to delve deeper into lNet’s workings, but can be useful for everyone as it explains the inner workings of events in the library. As mentioned in the previous chapter, lNet consists of Sockets, Eventers, Protocols and Sessions. This guide will look at the first two.

First things first tho.. what’s a handle? A handle is the absolute basic abstraction above OS handles, that is file descriptors, sockets and so on. On the cross-platform level, only sockets are handles (because Windows does not allow otherwise), however on Unix platforms, the eventers can monitor also file descriptors, pipes and possibly others too. So a handle (TLHandle) is the base class for a socket (TLSocket), and provides the “low level” abstractions above generic handles (not socket-specific yet). Handles also happen to be a double-linked-list.

A little side-note about sockets. Since they are handles, they already inherit all the stuff, but they also add their own. Namely socket-specific methods and data, but also ANOTHER double-linked list. The handle-level linked list data (next/previous) is for the eventers. The socket level linked list data (nextsock, prevsock) is for the protocols (TCP and UDP mainly).

1. ALL sockets ARE handles, but NOT all handles must be sockets.

2. Handles are owned by the eventer, but created elsewhere (in case of sockets, by the protocols. The creator property of TLSocket is used to store who made the socket)

The eventer is used to handle all the handles and their events. It’s the object which watches all of them for events (OnReceive etc. are ultimately called by the eventer). There’s one abstract eventer and multiple concrete eventers for specific OSes. The idea is to use the best possible candidate function on given OS (decided runtime, so no need to compile for linux version xxx). For example, on linux the best candidate is the Epoll() function, but it only exists on kernel 2.6+. On FreeBSD the KQueue() is used. The common, but low performance version is the Select() which exists on just about everything. All the specific eventers you will find are just implementations above specific OS functions to get the best out of it.

For example, while the select eventer has to cycle through all sockets each iteration, the epoll and kqueue eventers only have to go through those sockets on which something actually happened.

The eventer’s main method is the CallAction() in which all this monitoring is being done. All eventers watch for 3 basic events. Error, Write and Read. At this level, higher level socket-specific events like OnAccept or OnConnect are not yet known. The eventer then calls the handle’s OnXXX method (OnRead, OnWrite or OnError) and whatever handler is assigned there will handle the events.

In case of lNet’s protocols the handlers are inside TLConnection and TLTcp/TLUdp. It is these handlers that decide exactly what happened on the socket and call the higher level event callbacks like OnConnect or OnReceive.

The eventer mechanism however is completely independant of the rest of lNet. Neither handles nor eventers know about sockets or anything above them. The good thing is, that you can (on non-windows platforms) add your own handles to be watched to the same eventer which you use for your sockets (and you should, for performance reasons, eventers should be shared).

So say you’re writing a web server, and want to also watch your html/php/whatever files, and some pipes (fastcgi or such). All you have to do is to create TLHandles around the file descriptors, assign OnRead, OnWrite and OnError and add them to the eventer to be watched.

Sharing is good: (only for non-visual lNet! Visual lNet eventer is already auto-shared)

Sharing eventers is another important aspect of lNet. Imagine you have an elaborate program, or server which has multiple protocols. Say, raw TCP, HTTP and FTP all in one. The best thing to do to get the most performance is to share one eventer accross all these protocols, so ALL the sockets of ALL the protocols will be handles in one syscall. There are two ways to do this.

1. Create the eventer (with the BestEventerClass() metaclass returning function) BEFORE using any of the protocols (calling Listen or Connect on them). Then assign the eventer to them and start using them. Don’t forget to free the eventer on end (after freeing the protocols) as this is the “manual” method.

2. Activate any of your protocols (call Connect or Listen, no need to wait for OnXXX tho) and assign it’s eventer to all the other protocols. This method is “automatic” as it doesn’t require memory management (when the last protocol is freed, the eventer is freed).

Also note that you need to only call CallAction on one of the protocols (or directly on the eventer) once per iteration (if you’re on non-visual lnet of course), otherwise you’re just calling the same thing multiple times (Protocol.CallAction calls Eventer.CallAction)

45 Comments »

  1. Rajiv Ranjan said,

    I am usging TLTCPComponent component for a multiplatform Client application The problem here is sending work fine but Onreceive event i always get one character.

    am i missing some thing?

    • almindor said,

      So inside onReceive you do GetMessage(s) and the result is 1 with s containing one character? Do I get that right?

      • Rajiv Ranjan said,

        Yes its right,But i am sending more then one character, moreover the return value of the fuction getmessage is 22 , means i should have 22 character, but i always get one character in string s.

      • Rajiv Ranjan said,

        Resolved.As i was not formating my string properly after received.Thank you so much.

      • almindor said,

        Yes πŸ™‚ you need to filter for output formatting chars like line endings and 0 byte (which “terminates” a string in pascal).

        Your string actually HAS 22 bytes and length(s) should report it but memo code ends on 0s (I think because of pchar).

  2. Rajiv Ranjan said,

    procedure TForm1.Button1Click(Sender: TObject);
    begin
    talk.Connect(‘192.168.12.69′,9000);
    end;

    procedure TForm1.Button2Click(Sender: TObject);
    var lmsg,lbuf:string;
    begin

    lmsg:=sendText.text;

    talk.SendMessage(lmsg);
    end;

    procedure TForm1.Button3Click(Sender: TObject);
    begin
    hear.Host:=’192.168.12.69’;
    hear.Port:=9000;
    hear.Listen();
    end;

    procedure TForm1.Button4Click(Sender: TObject);
    begin
    hear.SendMessage(sendText1.text);
    end;

    procedure TForm1.FormCreate(Sender: TObject);
    begin

    end;

    procedure TForm1.HearAccept(aSocket: TLSocket);
    begin
    memo2.Append(‘HEAR accept to’ + aSocket.PeerAddress);
    end;

    procedure TForm1.HearCanSend(aSocket: TLSocket);
    begin
    Button4Click(self);
    end;

    procedure TForm1.HearConnect(aSocket: TLSocket);
    begin
    memo2.Append(‘HEAR connect to’ + aSocket.PeerAddress);
    end;

    procedure TForm1.HearDisconnect(aSocket: TLSocket);
    begin
    memo2.Append(‘Hear disconnect to’ + aSocket.PeerAddress);
    end;

    procedure TForm1.HearError(const msg: string; aSocket: TLSocket);
    begin
    memo2.Append(‘HEAR error ‘);
    end;

    procedure TForm1.HearReceive(aSocket: TLSocket);
    var ret:integer;
    var s:string;
    begin
    ret:= aSocket.GetMessage(s);

    if ret>0 then
    begin
    memo2.Append(‘HEAR received’);
    memo2.Append(s);
    end;

    end;

    procedure TForm1.TalkAccept(aSocket: TLSocket);
    begin
    //
    memo1.Append(‘TALK accept ‘);
    end;

    procedure TForm1.TalkConnect(aSocket: TLSocket);
    begin
    memo1.Append(‘TALK connect to’ + aSocket.PeerAddress);
    end;

    procedure TForm1.TalkDisconnect(aSocket: TLSocket);
    begin
    memo1.Append(‘TALK disconnect to’ + aSocket.PeerAddress);
    end;

    procedure TForm1.TalkError(const msg: string; aSocket: TLSocket);
    begin
    memo1.Append(‘TALK error ‘);
    end;

    procedure TForm1.TalkReceive(aSocket: TLSocket);
    var s:string;
    var ret:integer;
    var ss:string;
    begin
    //
    ret:= aSocket.GetMessage(s);

    if ret>0 then
    begin
    memo1.Append(‘TALK received’);
    memo1.Append(s);
    end;

    end;

  3. Rajiv Ranjan said,

    Hi almindor,

    Thanks for all your support.Now I have two queries.

    1)Previously I have used visual component TLTcpcomponent in lazarus window and all works well. Now I will be using non visual TLTcp for MAC OS using lazarus.

    2)OnReceived event is asyncronous and nonbloking or I need to use thread for receiving.

    Please suggest me on these two points.

    Thanks in advance
    Rajiv

    • almindor said,

      1. Yes, that’s currently the only option unless you can drop Carbon/Cocoa and compile for Gtk2 or Qt on OSX (in which case visual lnet works fine).

      2. It’s non blocking but you need to do a .CallAction if using non-visual lnet (see examples).

      • Rajiv Ranjan said,

        Thanks.
        I have another problem that ,If I use TLTcp insted of TLTcpcomponent .I always get socket error.Same code just the class is different means I am using TLTCP insted of TLTcpcomponent .

        Do i need to do some thing differently for TLTCP component?

      • Rajiv Ranjan said,

        Hi almindor ,

        Understood that after connect call i need to call callaction which makes the on connect fired.

        Some thing like this
        talk.Connect(‘192.168.12.69’,9000);
        repeat
        talk.CallAction; // wait for “OnConnect”
        until talk.Connected;

        my onconnect get fired all works well.
        Here also if i am using a GUI application i need to have some time out other wise if my server is not running the client application stuck in the repeate loop.

        Second thing is after sending some message here from server my on receive event doesn’t get fired.

        do i need to call again callaction menthod, if yes when?

        Rajiv

      • almindor said,

        You need to issue callaction periodically. It is this call which enables ALL events on lNet to work. Whenever you call .CallAction you enable lNet to ask the OS “what’s happening on my sockets” and act accordingly (which in turn calls OnReceive etc.)

        Depending on your main loop, you can either call CallAction each iteration non-blocking (timeout = 0). This is good if your main loops is iterating all the time in fairly short intervals. If you need to “sleep” (until things happen on the net) then you can use .Timeout property > 0 or -1 if you want to block indefenetly. It’s all a matter of how your main program works. You can even put it all in it’s own thread (but beware data corruption from other threads then).

  4. Rajiv Ranjan said,

    Hi almindor ,

    Thanks again for your descriptive explanation.
    I am writing a GUI based client application, which will communication with a server application.
    So i don’t have main loop as such , I am using a timer of 1000 ms to call “callaction” function.Currently it is solving my need.

    Do you think there should be any contradiction using timer?
    I am devolving a GUI client application in Lazarus MAC OS.

    Rajiv

    • Almindor said,

      You can use a TTimer but you should know the limitations:
      1. It’s going to cause latency (because you waste the time between). Depending on your network requirements this may not be an issue.
      2. Trying to fix that by making the interval lower will increase CPU use.

      There is another solution. You can use OnIdle (but be careful to set lNet’s TimeOut to 0). This means that whenever your app is not busy, networking code will get called and you’ll receive data.

      • Ryan Davis said,

        I’m also on OS X implementing lNet TCPIP with the non-visual components.

        I, too, was trying to figure what is the best well to “loop” CallAction. Because I read this post before implementing I tried implementing using OnIdle first. This does not seem to work well.

        If the application has focus it is faster, but can still lag. However, for some reason if the application does not have focus it is very slow with lag in seconds.

        However, using a TTimer with even an interval of 10 ms results in only 1-2% cpu usage for me. Granted I’m on a decently powerful machine, but from my testing that seems like the way to go, unless you have a third option or until you implement the visual components for Mac OS X ( please, please πŸ˜‰ )

      • almindor said,

        Using a TTimer is OK if it’s a client application (not a server) and you don’t expect DDOS attacks πŸ˜€

        I still don’t own any Mac hardware so the visual client on Mac is unlikely. Things might change in a few months tho πŸ™‚

  5. Rajiv Ranjan said,

    Thanks once again for your suggestion.

    If i leave TTimer apart, the other option I have is Thread.But thread have its own complications.

    I will try to use application.onidle if necessary.

    Rajiv

  6. John vd Waeter said,

    Hi,

    In a server (non-visual, written in Lazarus) I have the TLTCP-component.
    I stress-test the server (using a couple of clients that connect/disconnect at high rate). They connect, wait 250ms, send a little string, wait 250 ms, and disconnects. After 250 ms the process repeats.

    The server does a writeln to the console inside the OnConnect event and inside the OnDisconnect-event. In this writeln I show TLUCP.Count.
    In the onError-event a writln shows the error-message.
    I can beautifully see the count go up and down, perfectly matching the number of clients+1.

    But as soon as clients are 10 or more, TLTCP.Count only counts up, no longer down. If I then disconnect some clients, the number goes down again.

    If I then kill all clients, the number stays sticky at some arbitrary number (9, 10, 20..) or even a negative number.

    What can be the case ? There are no error-events fired.

    tia!
    John

    • almindor said,

      Hmm that’s probably a bug. Can you send me the test code please? Also specify what OS and platform you used.

      Thanks

  7. John vd Waeter said,

    Hi Almindor,

    I think you only need the server part? It’s written in Lazarus and runs on Debian.

    I’ll strip the code to the necessary part and let you know!

    John

  8. John vd Waeter said,

    Sent zip by PM.

  9. John vd Waeter said,

    oops, delivery failure….
    gmail don’t like zips with exe in it. I’ll try again without the exe

  10. Alfredo Gloria said,

    I working on a Server APP that must run on WinCE 6.0, the server needs to receive a string about 100 chars, all works fine except that the server app is accepting only 1 client, I’m using TLTCPComponent as server on WinCE app, the client is a hardware device (not a PC) using standard sockets, this is what appends:

    When first client connects OnAcept event is fired on server, and count property is incresed by 1, every time client sends data the onReceive event is fired and i get data transmited by client.

    when second client connects OnAcept event is never fired and count property is not incresed, if client sends data it never fires the onReceive event, but client don’t get any connection reject, the client thinks it is connected!

    When first client disconnects by any reason and tries to reconnect, OnAcept event is never fired, just happends the same as with the second client.

    to make server to accept a client again i need to call Disconnect and then listen method again, after that the first client will connect again.

    I have tested the TESTNET example on lnet/examples/visual/tcpudp and it does the same, it only accepts 1 client, any idea?

    • almindor said,

      This looks like a bug. Perhaps some changes in winCE fpc code or lazarus code has broken event handling for accepts. Could you try a console application somehow? If that works then it’s the lNet lcl event handler which causes this. I currently don’t have Windows or WinCE emulator anywhere so I’m not sure when I could try to find the problem. I’ll try to get the WinCE emulator working inside a winXP VirtualBox install.

    • Robin Hoo said,

      Pls check out the bug fix of winsocket.pp of post http://www.lazarus.freepascal.org/index.php/topic,18913.0.html

      • almindor said,

        Replied in the thread

  11. Rajiv said,

    I have a simple problem here, I have Client server application,Where client can connect to many server.Suppose if type wrong server credential then i have to disconnect to first server and connect to new server.How do i clean my first socket connection.my problem is when i call socket.disconnect i don’t receive disconnection event and when i try to connect to new server i receive first disconnection.Please help

  12. Claudio said,

    Hi, is there any way to accept only 3 connections and then stop listening ?
    Something like this:


    tcp.Listening(Port);

    onAccept event
    if tcp.Count = 3 then “tcp.StopListening”,

    I need to restrict connections to only 3.

    Thanks in advance.

    • Logic_Bomb said,

      @Claudio:

      SInce you need to listen on that port in order to receive new messages from clients it’s not a good idea to stop listening, but you could do something like this:

      procedure TForm1.tcpaccept(aSocket: TLSocket);
      begin
      If tcp.count >= 3 then begin
      aSocket.Disconnect();
      end;
      end;

      Hope this helps =)

      Logic~

  13. Logic_Bomb said,

    @Claudio:

    SInce you need to listen on that port in order to receive new messages from clients it’s not a good idea to stop listening, but you could do something like this:

    procedure TForm1.tcpaccept(aSocket: TLSocket);
    begin
    If tcp.count >= 3 then begin
    aSocket.Disconnect();
    end;
    end;

    Hope this helps =)

    Logic~

  14. Josh said,

    how can i send a smtp message from the program without authentification to my mailadress (www.web.de)

    • almindor said,

      Unauthorized sending is usually only possible from internal networks, this is a question for the SMTP server admin, not lNet

  15. zandoye said,

    I’m using tLTcp with tLEventer. I’ve set tLTcp.eventer to the associated tLEvent object and call tLEvent.callAction to wait for new events. After a socket disconnected, OnDisconnect gets called shipped with an aSocket:tLSocket parameter.
    Should I call aSocket.destroy explicitly to prevent the disconnected aSocket from leaking the memory?
    And If it’s auto destroyed, will it occupy memory for a long time?

    • almindor said,

      You don’t ever call destroy on sockets. All memory is handled by lNet. Each socket is disposed after disconnect finishes from both sides (the event handler frees all completely disconnected sockets after the event reporting loop, which is usually after the OnDisconnect). If you find out that there’s a loop please do a bugrep (on fpc/lazarus bugrep page, choose packages/lnet) with some reproducable code.

  16. prof7bit said,

    I have a problem with the TLEpollEventer:

    I am using a TLTcp to connect to a socks4a proxy, in the connect event I send the socks4a request.

    In the receive event I receive the answer from the socks proxy, this is should be exactly 8 bytes.

    -> If the answer is ok (byte 2 = #90) then I know I am fully connected to the other end and can now use the connection like normal, I set all event methods to nil and pass the socket to another object (my own wrapper around such a connection) that will install its own event handlers for read and error. This works without problems.

    -> If the answer is not ok (not 8 bytes or any other code than #90 in byte 2) then i know the socks connection attempt failed, I now expect the socks proxy to disconnect immediately. Here begins the problem: The socks proxy WILL disconnect now immediately after the last byte and the epoll eventer will (sometimes!) enter a very strange state in the next iteration where it will cause memory corruption, poll invalid handles, call receive with 0 bytes a million times, etc. BUT *only* sometimes.

    I have tried all kinds of stuff after receiving the socks answer, either one of the following:
    * do nothing, just hope the disconnect will fire in the next iteration.
    * call Disconnect() on the socket.
    * call fpshutdown() on the handle.
    * call fpshutdown() and set Dispose:=True and do my own cleanup immediately myself because then no event will follow anymore

    It all does not matter, the epoll eventer will in one out of maybe 5, sometimes one out of 1000 times (depending on current moon phase) cause memory corruption when the other side disconnects. The select eventer does not cause these problems.

    My code is here: https://github.com/prof7bit/TorChat/tree/lnet/src/client
    the code for socks connect attempt is in tc_buddy.pas
    (other code using lNet (managing the connection after socks4a connect was successful) is in tc_conn.pas and seems to work without problems)

    Maybe I’m going to create a simpler example to provoke the error when I have some time. lNet looks very promising, it seems to be exactly what I need for such kind of software, otherwise I would have to implement all the select() stuff myself, I just can’t sleep well until I have understood what exactly is causing these problems here.

    • almindor said,

      I’ll look into it, in the mean time you can force lnet into using select eventer and try if it works. Just assign a new tlselecteventer to tltcp before listen/connect.

      • prof7bit said,

        I have tried to reproduce it with a simpler example and it seems to happen only when there are more than one eventer (each with its own thread). In my current architecture (the libpurple plugin I am writing) there can be multiple accounts at the same time which means for me I am creating multiple instancesof my TTorChat client object, each is completely autonomous, has its own eventer and eventer thread, its own listening port, its own buddy list with buddies (each buddy has an outgoing and an incoming TCP connection). The problem starts appearing if I start a second instance of TTorChat. But they don’t share any resources with each other. The only thing that comes to my mind is maybe one instance is getting OS handles assigned for new connections that were previously assigned to the other client and the other eventer has not yet forgotten about them somehow (?) maybe.

        My demo program to test this was therefore creating two TLTcp objects, two eventers, two eventer threads, and then rapidly making invalid socks requests (using randomly chosen one of the two client objects) to my running tor proxy (requests that would result in immediate disconnect) and *sometimes* it would indeed provoke this error, but it was far more difficult to reproduce it than it was with the full blown libpurple plugin. It almost seems like some sort of race condition but when looking at the code I have no idea where this might happen and how two eventers could ever interfere with each other.

        I am still trying to make a more reliable demo-application that demonstrates the problem more reliably. In my plugin I am now using the TLSelectEventer (for less likely surprises on different platforms) and this one works like a charm. My plugin does not need high throughput or thousands of connections, it only needs to work reliably without crashes and leaks.

      • almindor said,

        Thanks for the testing, at least select eventer works. I’ll try and analyze epoll with multiple threads. Btw. Lnet has an experimental tthreadedeventer whcich uses threadpools to work well with huge loads, you might want to check it out.

  17. prof7bit said,

    There is one little problem with the TLSelectEventer on Windows: The WinSock2 unit from FPC defines the FD_SETSIZE very low, only 64 sockets. This value was meant by microsoft to be changed prior to inclusion of the header file to a more reasonable value but obviously doing so is impossible in Pascal because its a precompiled unit and not an include file and so we now have to live with this unusably low default value.

    A more reasonable value would have been 1024 to make it consistent with the abilities of unix select().

    After thinking about it for a while I have come up with a very simple and non-invasive way to patch it: A unit that comes after another unit in the uses clause will override the definitions of the same name in the first unit. Here is my patch for lNet:

    https://github.com/prof7bit/TorChat/blob/torchat2/src/client/lnet-0.6.5/CHANGES.diff

    it changes only one line in osunits.inc and introduces a small new unit that overrides the relevant definitions in WinSock2.

    I have tested it on Windows XP and it works like a charm with 1023 open connections (one could easily go even higher with the number, I have tried up to 4000 but I think making it exactly equal to the select on the other platforms is the most reasonable thing)

    BTW is there a discussion forum or mailing list for lNet? This wordpress blog is a bit impractical for longer discussions and not very easy to find.

    • almindor said,

      Hmm this is a nice hack πŸ™‚ I’ll be sure to include it. As for the forums, you can use Lazarus forums, networking section, just include lNet in the subject. I go there from time to time.

  18. Bond Keevil said,

    I have a client/server application that implements a custom protocol using the TLTCP Component with a TLCLEventer eventer. The server code is completely encapsulated in TServer and the client code in TClient. Platform is Linux 64 / GTK.

    1. As long as these components are in a windowed application then everything works fine, but I now want my Server application to be a console mode application. When I create my TServer in the console application using roughly the same code, none of the events of TLTCP are ever fired. Any ideas?

    2. Is there a way to execute the TLTCP listen command in a blocking mode with a timeout? It makes more sense in console mode than an endless loop of Sleep(100) statements.

    • almindor said,

      You need to call eventer.CallAction in the main loop of your console app for it to work (see console examples).

      There are two ways to do blocking mode. One is to use -1 as the timeout value for the eventer. The other is to switch to blocking mode via options on the socket/component.

  19. I am trying to figure out how you can trigger on timeout, if the following code is my program.

    type
    TForm1 = class(TForm)
    net1:TLTCPComponent;
    Timer1:TTimer; //Set for 30 milliseconds
    procedure FormCreate(Sender:TObject);
    procedure net1receive(aSocket:TLSocket);
    procedure Timer1Timer(Sender:TObject);
    private
    public
    end;

    procedure TForm1.FormCreate(Sender:TObject);
    begin
    Timer1.Interval := 30; //milliseconds
    net1.connect(‘192.168.1.31’,10002);
    Timer1.Enabled := true;
    end;

    procedure TForm1.net1receive(aSocket:TLSocket);
    var
    rbyte:array[0..3] of byte;
    begin
    asocket.Get(rbyte, sizeof(rbyte));
    end;

    procedure TForm1.Timer1Timer(Sender:TObject);
    var
    bbyte: array[0..3] of byte;
    begin
    bbyte[0] := $FF;
    bbyte[1] :=$01;
    bbyte[2] :=$03;

    net1.send(bbyte, sizeof(bbyte), net1.iterator);
    end;

  20. Yuskamal said,

    when more than 1 client connected to server, can i send message to specified client ?

    • almindor said,

      Of course πŸ™‚

      When you get a connection accepted event you get a socket parameter in the handler. This is the socket you need to reply with. Same on messages coming in, they have a socket in the handler and you use that socket to reply directly to the sender.

  21. Nicolas said,

    Hi Almindor, I’ve developed a console tcp server to be run on Linux, based on lNet tcp server example. My server implements keepalives (Because clients are mobile devices).

    When keepalives aren’t received (Timeouted) my server disconnects the socket (Using “aSocket.Disconnect”), but the connections are kept on FIN_WAIT_1/FIN_WAIT2 states and file descriptors are not freed. Eventually my app crashes because server was out of FD.

    Instead, if I use “aSocket.Disconnect(True)” (To force disconnecting socket) the FD are freed, but memory leaks occur (My servers grows on memory usage).

    Am I missing something? Thank you!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: