EQEmulator Forums - perl quests calling c functions

EQEmulator Forums (https://www.eqemulator.org/forums/index.php)

- Archive::Development (https://www.eqemulator.org/forums/forumdisplay.php?f=621)

- - perl quests calling c functions (https://www.eqemulator.org/forums/showthread.php?t=11387)

perl quests calling c functions

One of the problems with calling a perl quest is that once the script is entered, it lacks the ability to interact with the server that called it. I have a way to fix this, but before I go through the trouble of typing it all out, I'd like to hear feedback.

The basic rundown is this:

1. Create the glue code in c/c++ to tie the zone server and perl together.
2. Create a shared library and a perl module to access it.

Here's one test script I've been using:

Code:

use Mytest;



sub EVENT_SAY {

  if ($text =~ /Hail/) {

    quest::say("Me hopes you gots yer club.");

    if (&Mytest::hasItem($charid, 6621)) {

      quest::say("HAR HAR YOU GOTS IT!");

    }

    else

    {

      quest::say("Dat tew BAD. Yew gots no clubs!");

    }

  }

}

BTW, there is also a new exported variable in this example, $charid.

What I'd like to know is if someone (Eglin?) is already working on this, or has a better way to do it. Anyway, if you want more info or not, let me know.

I know I'd like to see it and I'm sure there are others. It would be nice to have a standard library of functions like that to distribute with the rest of the package.

Here is the process to set this stuff up in case anyone wants it. I was really posting more to the point to find out if anyone saw a problem with calling functions from perl this way. I only suggest it because from what I can tell, it's the best method. I'm also soliciting for any functions anyone might need to call from perl.

First, the code for the functions we will be calling from perl:

Code:

/* cperl.cpp */



#ifndef CPERL_CPP

#define CPERL_CPP



#ifdef EMBPERL



#include "masterentity.h"



extern Database database;



extern "C" {



  bool hasItem(unsigned charid, unsigned itemid)

  {

    Inventory inv;

    int ret = false;



    if (database.GetInventory(charid, &inv))

       if (inv.HasItem(itemid) != -1)

          ret = true;



    return ret;

  }



}





#endif // EMBPERL



#endif // CPERL_CPP

I also exported a new variable to export to perl, charid. This is around line 94 in embparser.cpp:

Code:

if (client && npc) {

                        fac = client->GetFactionLevel(client->GetID(), npcmob->GetID(), client->GetRace(), client->GetClass(), DEITY_AGNOSTIC, npc->GetPrimaryFaction(), npcmob);

                        ExportVar(packagename.c_str(), "charid", itoa(client->CharacterID()));

                }

Next, run make to compile the object file for cperl.cpp, and recreate the zone binary:

Now that we have our object code, we need to use XS to bring it and perl together. The following will create a work area for that:

Code:

h2xs -A -n Mytest

This will create a Mytest directory with a few skeleton files. We will now edit Mytest.xs. XS is a sort of language of its own. This is what my Mytest.xs file looks like:

Code:

#include "EXTERN.h"

#include "perl.h"

#include "XSUB.h"



#include "ppport.h"



MODULE = Mytest         PACKAGE = Mytest



bool

hasItem(charid,itemid)

    INPUT:

        unsigned charid

        unsigned itemid

    CODE:

        RETVAL = hasItem(charid,itemid);

    OUTPUT:

        RETVAL

Now, run "perl Makefile.PL". This will create a new makefile.
Run "make". A file called Mytest.o will be generated, along with blib/lib/Mytest.pm. Copy the Mytest.pm file to your root Mytest directory.

We need to link Mytest.o and cperl.o together, so:

Code:

ld -shared `perl -MExtUtils::Embed -e ldopts` -o Mytest.so Mytest.o cperl.o

For good measure, I like to run "ld -shared -E Mytest.so" to see if there are unresolved symbols. The shared library should be created. Add the root Mytest directory to your perl INC path (ie: you can do this by setting the PERL5LIB environment variable). The perl script I tested this with is included in the first post. Go back to the eqemu directory and run world and zone. You should be able to log in, hail whoever you assigned the script to, and have the correct output.

edit: I forgot to note, make sure the new cperl.o (or whatever you call it) is put into the zone makefile so it gets compiled into the zone binary. Also, all of this does NOT need to be done to set it up on a server. Once the source from XS is created, that can be incorporated by itself (compiled and linked with the cperl.o file). This is just to quench any curiosity over this issue.

It should be possible to transfer execution back and forth between the host app and the embedded interpreter without writing an external module. Look at pickle for examples of how to do this in an elegant fashion (the *_arg stuff requires no .so, IIRC). I said somewhere else that I think the best thing to do is to write a single glue function to flush the command queue and I'm sticking with my assertion. If you're going to write some XS, I'd love to see you write a little function that flushes the command queue... I've seen some _extremely_ ugly and hacktastic workarounds for not having a method of transferring execution (like the flag stuff) that will have to be reworked completely and I guess that more will come unless a better method is made avaliable. If your timetable allows you to write this before me, then that would be great.

edit: Oy... ignore this post. Things appeared to work, but it was actually loading the module from disk, not from the eval :( ... There are troubles bootstrapping purely from using evals. Currently, I can call C functions from the scripts without needing a shared library, but there still needs to be a whatever.pm file to help out.

alright, posting a reply here because i think the thread is the right one.

It naturally bumps from these threads, that i don't want ot go off topic
http://www.eqemulator.net/forums/vie...4b1e04e71c133e
http://www.eqemulator.net/forums/vie...4b1e04e71c133e

The general perl process in EQEMu atm is having a callback called when an event is triggered, and this callback does some perl code running, and pushes one or more commands on a queue. Commands are then processed from the queue, after the perl callback has returned. Is that correct ?

Unless big rework is done of the perl quest system, there is little chance to go into calling C from perl (appart some stand-alone library modules), as this would lead to uncontrollable loops or deadlocks, and such stack switching is additional overhead.

The current process leads to exporting more variables as quests require them. It can be optimized, but should'nt be workaround'd

This is just personnal opinion, and i like to get more of your forethoughts on perl quest system.

I am hesitant to say that perl currently "calls back" to C. This is the process, which I think you have right:

-server calls perl function
-perl function (ie: EVENT_SAY) adds functions/variables to cmd_queue
-perl function ends
-server runs through cmd_queue, processing each function

So, there are two big reasons for callbacks: First, to call C functions and get data back immediately; second, to get rid of a lot of overhead.

I actually just submitted a patch that deals with the command queue here: http://www.eqemulator.net/forums/viewtopic.php?t=13906. It effectively makes SendCommands() a re-enterable function.

As far as perl calling C code, I have done this. It's been a little bit since I have looked at it, so I am a bit fuzzy, but involves XS to make the glue between perl and C. I have been able to incorporate just about all of this into the server code itself, so no external library is needed.

However, one of the things I have done is inject the "require MODULE.pm;" statement into a C perl-eval (needed so perl knows how to use package). This in turn tries to load MODULE.pm from disk. I have not been able to find a way to completely hide this in the server code. As things are now, it is all in the server code, except for that external file.

I am again looking through the different perl man pages. I think the answer may be in perlcall, which I may have overlooked before. It seems there is another way to do this. By all means investigate and try things. I am still not comfortable with all this Perl/C/XS stuff. I have only been trying to figure it out because I think it would be a huge boon to perl quests, and nobody is really doing anything with it.. :wink:

In case you have not seen these, I offer some references that I have used:

man perlembed
man perlxstut
man perlxs
man perlcall
http://search.cpan.org/~jtobey/pickle-0.5.1/

I have tried to keep the posts central, but I have done a poor job of it. :cry: I am usually on the eqemu irc server in #ProjectEQ and #npcmovdb lately, nickname bleh if you are interested in that.

yup, thanks for quick reply. :)

i get to most of your views.

We'll prolly talk on irc ; well i'm away for a few days now on, but sure will soon.

Just made a post to http://www.eqemulator.net/forums/viewtopic.php?t=13920. As of writing I am the only person to test this out. It all seems to work, though.

Worked that on a bit :

- .cpp file is generated by XS directly. This is handled by the makefile
- xs makes function mapping really easy. To map a C function a quest function, simply put the prototype in the cperl.xs file
- more complex functions (like those that refer to instances, not C functions), require XS writting.

Here's what's added to the makefile :

Code:

PERLMOD=/usr/lib/perl5/5.8.0/ExtUtils/

XSUBPP=perl -MExtUtils::Miniperl /usr/bin/xsubpp -C++ -prototypes -nolinenumbers -typemap $(PERLMOD)/typemap



%.cpp: %.xs

  $(XSUBPP) -typemap EQEMu.typemap $&lt; > $@

of course you need to add the cperl.o to your SF files.

now here is the cperl.xs file :

Code:

#include "masterentity.h"



#include "EXTERN.h"

#include "perl.h"

#include "XSUB.h"



// more definitions for the C functions allowed here

const char *getItemName(unsigned itemid)

{

    const Item_Struct* item = NULL;

      item = database.GetItem(itemid);



        if (item)

              return item->Name;

          else

                return NULL;

}





MODULE = EQEMu PACKAGE=eqemu



const char *

getItemName(unsigned itemid);

In the above example, a new function was created, and mapped to perl. It's a bit long because it's a new function. Else, the function body does not need to be in this file. Only the part after the MODULE keyword is to change.

i also added a EQEMu.typename file, that is empty atm, to allow mapping of C types to perl types. Leave empty for now.

There are several possibilities with XS, that allow to really tweak the interface between C and Perl. Full descriptions are :
a tutorial
reference of XS constructs

Note, as compared to m0ni9's work :
- changed MODULE name from qc to EQEMu. Less risks to cope with an existing or new perl module
- changed package name from quest to eqemu. This is important, because in perl code, you are going to call both subs :

Code:

sub EVENT_ITEM { 

  quest::say("Ooh, a " . eqemu::getItemName($item1) . "! Here's your reward!"); 

  quest::givecash(5,0,0,0); 

  quest::exp(100); 

}

The quests::thing() is a command, pushed on the queue, that the zone server will process after the perl EVENT subroutine is done
The eqemu::thing() is called immediatly in perl subroutine. Thus it is effective to write :

Code:

my $itemname=eqemu::getItemName($item1);

I would not explain more about XS now (because i'm just not able to, i discovered it thanks to mOoni9 :)). Just some info to add more functions :
- if function exists, simply put prototype in the second part. Insert headers required by function prototypes on top of file (near #include "masterentity.h")
- append prototypes in second part of the file, after the MODULE construct. You need not to repeat the MODULE line, just add prototypes
- XS is touchy about indentation. To write a prototype : return type must be on first line, alone. It must have no leading spaces. Function name and args can be ANSI style, or trditionnal C style. In latter case, indent parameter declaration, that must have leading spaces ; You'll get more details there

happy coding ! Thanks m0oni9 again for the pull.

p.s. This post might be edited more than a couple of times on, this is very pre-alpha :)

Just a note (and thanks for checking this stuff out also, smogo).. It is probably a good idea to have any new XSUBs use the quest package also. This will keep any quests that rely on current commands compatible in case there are any conversions. Of course if they are using exported variables instead of to-be getVariable calls, it can't be done for that.. :wink:

Anyone got time to try that out and add new functions ?

i just don't know what could be usefull, like :
quest::getTimeOfDay()
quest::openDoor($doorid);
quest::loc()

... or is it talking Chinese ?

I'm going to be playing around with this...