NAVIGATION

Calling Foreign Functions

One of the most important features that a practical programming language must support is the ability to call functions written in other languages. There are too many useful libraries written in the established languages to consider rewriting them in another programming language. Stanza provides support for calling any function using the calling convention for the C programming language. This means that you can use any library written in C, or that provides a C interface, in Stanza. Since the dominant consumer operating systems today use a C calling convention, this means that the vast majority of libraries can be called from Stanza. This chapter will show you how.

Writing a C Function

Here is a fibonacci function written in C. Create a fibonacci.c file with the following contents.

#include<stdio.h>
#include<stdlib.h>

int generate_fib (int n) {
   int a = 0;
   int b = 1;
   while(n > 0){
      printf("%d\n", b);
      int c = a + b;
      a = b;
      b = c;
      n = n - 1;
   }
   return 0;
}

int main (int nargs, char** argvs) {
   generate_fib(10);
   return 0;
}

Compile and run the above program by typing

cc fibonacci.c -o fibonacci
./fibonacci

in the terminal. It should print out

In the next step, we will call the generate_fib function from Stanza.

Calling our C Function

The first step is just to remove the main function in fibonacci.c since the program is now being initialized and driven by Stanza.

Next create a file named fib.stanza with the following contents.

defpackage fib :
   import core
   import collections

extern generate_fib: int -> int

lostanza defn call-fib () -> ref<False> :
   call-c generate_fib(10)
   return false

println("Calling fibonacci")
call-fib()
println("Done calling fibonacci")

To compile both the fib.stanza and fibonacci.c files together, and run the program, type the following into the terminal.

stanza fib.stanza -ccfiles fibonacci.c -o fib
./fib

It should print out

Calling fibonacci
1
1
2
3
5
8
13
21
34
55
Done calling fibonacci

Thus our Stanza program successfully calls and returns from the generate_fib function written in C. Let's go through the program step by step.

Declaring an External Function

The line

extern generate_fib: int -> int

declares that there is a function defined externally called generate_fib that takes a single integer argument and returns a single integer argument.

Notice that int is not capitalized. This is important. int refers to the LoStanza integer type, and is different from the Stanza type Int. We'll go over what this means later.

Let us suppose that generate_fib took two arguments instead of one. Make the following change to the generate_fib function, where it now accepts an argument, b0, to indicate the initial value of b.

int generate_fib (int b0, int n) {
   int a = 0;
   int b = b0;
   ...
}

Then the extern statement, and the call to generate_fib would have to be updated accordingly.

extern generate_fib: (int, int) -> int

lostanza defn call-fib () -> ref<False> :
   call-c generate_fib(2, 10)
   return false

Compiling and running the new program now prints out

Calling fibonacci
2
2
4
6
10
16
26
42
68
110
Done calling fibonacci

Declaring a LoStanza Function

LoStanza is a small sub-language within Stanza that allows users to precisely specify data layouts and perform low-level hardware operations. LoStanza can be used for writing high performance code, communicating with external peripherals, and implementing system level functions. Stanza's garbage collector, for example, is written in LoStanza. In this chapter, we are using it to interface with externally defined functions.

The line

lostanza defn call-fib () -> ref<False>

declares a LoStanza function called call-fib. Its return type, ref<False>, indicates that it returns a reference to the Stanza type, False.

The line

call-c generate_fib(10)

calls the generate_fib function with the argument 10. The call-c tells Stanza to call generate_fib with the C calling convention. By default, Stanza uses the Stanza calling convention to call other functions, and if you forget the call-c it will seriously confuse generate_fib and crash the program.

Finally, the line

return false

simply returns false to whomever called call-fib.

C Functions that Return void

When a C function is declared to return a value of type void, it means that the function is called for its side effects only, and returns an arbitrary value. Let's change generate_fib to return void.

void generate_fib (int b0, int n) {
   int a = 0;
   int b = b0;
   while(n > 0){
      int c = a + b;
      printf("%d\n", c);
      a = b;
      b = c;
      n = n - 1;
   }
}

Stanza does not provide any explicit support for modeling arbitrary values, so the extern statement would remain as

extern generate_fib: (int, int) -> int

and, as the programmer, you would have to remember (or document) that generate_fib returns some random integer that should not be used.

Calling LoStanza from Stanza

The arguments to generate_fib are currently hardcoded into the call-fib function. Let's change that to allow both b0 and n to be passed as arguments to call-fib.

extern generate_fib: (int, int) -> int

lostanza defn call-fib (b0:int, n:int) -> ref<False> :
   call-c generate_fib(b0, n)
   return false

And our test code will now call call-fib with different arguments.

println("Calling fibonacci(1, 10)")
call-fib(1, 10)
println("Calling fibonacci(2, 10)")
call-fib(2, 10)
println("Done calling fibonacci")

However, attempting to compile the above gives us the following error.

LoStanza function call-fib of type (int, int) -> ref<False> 
can only be referred to from LoStanza.

As mentioned, int is a LoStanza type, and you're not allowed to call it directly from Stanza with Stanza objects.

Convert Stanza Objects to LoStanza Values

The type Int is declared like this.

lostanza deftype Int :
   value: int

We will explain what that means in more detail later, but for now, notice that it contains a field called value that is of type int. Thus, we will modify our call-fib function to accept references to Int objects, and then pass their value fields to generate_fib.

lostanza defn call-fib (b0:ref<Int>, n:ref<Int>) -> ref<False> :
   call-c generate_fib(b0.value, n.value)
   return false

With this change, the program now compiles correctly, and prints out

Calling fibonacci(1, 10)
1
1
2
3
5
8
13
21
34
55
Calling fibonacci(2, 10)
2
2
4
6
10
16
26
42
68
110
Done calling fibonacci

A LoStanza function can be called from Stanza if and only if all of its argument types and return type are ref<T>, indicating that it accepts and returns a reference to a Stanza object. LoStanza functions that can be suitably called from Stanza are indistinguishable from regular Stanza functions. So in addition to being called directly, they can also be passed as arguments, and stored in datastructures.

Convert LoStanza Values to Stanza Objects

Let us now change generate_fib to return the n'th fibonacci number, instead of printing all of them.

int generate_fib (int b0, int n) {
   int a = 0;
   int b = b0;
   while(n > 0){
      int c = a + b;
      a = b;
      b = c;
      n = n - 1;
   }
   return b;
}

We'll also update our call-fib function to return the result of generate_fib.

lostanza defn call-fib (b0:ref<Int>, n:ref<Int>) -> int :
   val result = call-c generate_fib(b0.value, n.value)
   return result

Here's the updated test code that prints out the result of calling call-fib.

println("fibonacci(1, 10) =")
println(call-fib(1, 10))
println("fibonacci(2, 10) =")
println(call-fib(2, 10))
println("Done calling fibonacci")

However, attempting to compile the above gives us this familiar error.

LoStanza function call-fib of type (ref<Int>, ref<Int>) -> int 
can only be referred to from LoStanza.

As mentioned already, a LoStanza function can be called from Stanza if and only if all of its argument types and return type are ref<T>. We learned how to convert Stanza Int objects into LoStanza int values in the previous section. Now we'll learn how to convert LoStanza int values back into Stanza Int objects.

To create a Stanza Int object, we use the LoStanza new operator.

lostanza defn call-fib (b0:ref<Int>, n:ref<Int>) -> ref<Int> :
   val result = call-c generate_fib(b0.value, n.value)
   return new Int{result}

Our test code now compiles and runs, and prints out

fibonacci(1, 10) =
89
fibonacci(2, 10) =
178
Done calling fibonacci

Note that the LoStanza new operator is completely different than the Stanza new operator. It is best to consider LoStanza as a completely separate language from Stanza. It has its own syntax, operators, and behaviour. The thing that makes LoStanza unique is that there is a well-defined and flexible interface between it and Stanza.

LoStanza Types

There are a handful of additional LoStanza types in addition to the int type that we used in the declaration of the generate_fib function.

Primitive Types

Here is a listing of the rest of the LoStanza primitive types, along with an example of their values.

val x:byte = 42Y
val x:int = 42
val x:long = 42L
val x:float = 42.0f
val x:double = 42.0

A byte is an 8-bit unsigned integer. An int is a 32-bit signed integer. A long is a 64-bit signed integer. A float is a 32-bit single precision floating point number. And a double is a 64-bit double precision floating point number.

The above primitive types have an associated Stanza type, each declared to contain a single value field containing the LoStanza representation of its value. The associated Stanza types for byte, int, long, float, and double, are Byte, Int, Long, Float, and Double, respectively. In addition to Byte, the Stanza type Char is also declared to contain a single value field of type byte.

As an example, let us write a version of generate_fib that works on double precision floating point numbers.

double generate_fib_d (double b0, int n) {
   double a = 0.0;
   double b = b0;
   while(n > 0){
      double c = a + b;
      a = b;
      b = c;
      n = n - 1;
   }
   return b;
}

Here is the LoStanza code needed to be able to call generate_fib_d from Stanza.

extern generate_fib_d: (double, int) -> double

lostanza defn call-fib (b0:ref<Double>, n:ref<Int>) -> ref<Double> :
   val result = call-c generate_fib_d(b0.value, n.value)
   return new Double{result}

Now armed with double precision floating point, let's calculate the 100'th fibonacci number.

println("fibonacci(1.0, 100) = ")
println(call-fib(1.0, 100))
println("fibonacci(2.0, 100) = ")
println(call-fib(2.0, 100))
println("Done calling fibonacci")

Compiling and running the above prints out

fibonacci(1.0, 100) = 
573147844013817200640.000000000000000
fibonacci(2.0, 100) = 
1146295688027634401280.000000000000000
Done calling fibonacci

Notice that the call-fib function is overloaded to accept both Int and Double arguments. LoStanza functions have all the same features as Stanza functions, and this includes their ability to be overloaded.

Pointer Types

Pointers are represented in LoStanza with the ptr<t> type. The little t represents any LoStanza type. For example, here is the type representing a pointer to an int,

ptr<int>

and here is the type representing a pointer to a pointer to an int,

ptr<ptr<int>>

The type

ptr<?>

represents a generic pointer to anything.

As an example of their use, let's call the C malloc and free functions to allocate and delete space for three integers.

extern malloc: long -> ptr<?>
extern free: ptr<?> -> int

lostanza defn try-pointers () -> ref<False> :
   val ints:ptr<int> = call-c malloc(3 * sizeof(int))
   call-c free(ints)
   return false

The [] operator in LoStanza is the dereference operator and retrieves the value stored at the given pointer address. Here is an example of storing and retrieving values into and from the ints pointer.

lostanza defn try-pointers () -> ref<False> :
   val ints:ptr<int> = call-c malloc(3 * sizeof(int))
   [ints] = 42
   [ints + 4] = 43
   [ints + 8] = 44
   val x = [ints]
   val y = [ints + 4]
   val z = [ints + 8]
   call-c free(ints)
   return false

Programmers familiar with C should note that arithmetic on pointers do not automatically operate in terms of the size of the pointer's data type. To retrieve the i'th element from a pointer, assuming that its elements are stored contiguously, we use the following syntax.

lostanza defn try-pointers () -> ref<False> :
   val ints:ptr<int> = call-c malloc(3 * sizeof(int))
   ints[0] = 42
   ints[1] = 43
   ints[2] = 44
   val x = ints[0]
   val y = ints[1]
   val z = ints[2]
   call-c free(ints)
   return false

This is equivalent to the previous example.

Declaring a LoStanza Type

Consider the following definition of the C type Point3D and function get_origin.

typedef struct {
   float x;
   float y;
   float z;
} Point3D;

Point3D* get_origin () {
  Point3D* p = (Point3D*)malloc(sizeof(Point3D));
  p->x = 0.0f;
  p->y = 0.0f;
  p->z = 0.0f;
  return p;
}

Point3D is a struct that contains three float fields, and get_origin returns a pointer to a Point3D.

Here is how we would declare our own LoStanza type to mirror the C type definition.

lostanza deftype Point3D :
   x: float
   y: float
   z: float

Here's a function that demonstrates calling get_origin and returning the x field in the returned point.

extern get_origin: () -> ptr<Point3D>

lostanza defn origin-x () -> ref<Float> :
   val p = call-c get_origin()
   return new Float{p.x}

Here's some code to test the origin-x function.

println("The x coordinate of the origin is %_." % [origin-x()])

which prints out

The x coordinate of the origin is 0.000000.

Reference Types

A reference to a Stanza object is represented with the ref<T> type. The big T represents any Stanza type. We've already used the ref<Int>, and ref<Float> types in our examples.

Our previous function origin-x returned the x coordinate of the origin. But we would really like to just return the entire point to Stanza. Similar to how we converted int values to Int objects, this is done using the new operator.

lostanza defn origin () -> ref<Point3D> :
   val p = call-c get_origin()
   return new Point3D{p.x, p.y, p.z}

And here are the LoStanza getter functions for a Point3D that allows Stanza to retrieve the coordinates within it.

lostanza defn x (p:ref<Point3D>) -> ref<Float> :
   return new Float{p.x}
lostanza defn y (p:ref<Point3D>) -> ref<Float> :
   return new Float{p.y}
lostanza defn z (p:ref<Point3D>) -> ref<Float> :
   return new Float{p.z}

Here's some code to test our new origin function.

val p = origin()
println("The x coordinate of the origin is %_." % [x(p)])
println("The y coordinate of the origin is %_." % [y(p)])
println("The z coordinate of the origin is %_." % [z(p)])

Compiling and running the above code prints out

The x coordinate of the origin is 0.000000.
The y coordinate of the origin is 0.000000.
The z coordinate of the origin is 0.000000.

As one last example, let's write, in LoStanza, a constructor function for Point3D objects that can be called from Stanza.

lostanza defn Point3D (x:ref<Float>, y:ref<Float>, z:ref<Float>) -> ref<Point3D> :
   return new Point3D{x.value, y.value, z.value}

Here's some test code for trying out our constructor function.

val p2 = Point3D(1.0f, 3.4f, 4.2f)
println("The x coordinate of p2 is %_." % [x(p2)])
println("The y coordinate of p2 is %_." % [y(p2)])
println("The z coordinate of p2 is %_." % [z(p2)])

which, when compiled and ran, prints out

The x coordinate of p2 is 1.000000.
The y coordinate of p2 is 3.400000.
The z coordinate of p2 is 4.200000.

With these definitions, Point3D becomes a type that we can freely manipulate from Stanza. We can create Point3D objects, and we can retrieve its fields.

Literal Strings

A literal string in LoStanza has type ptr<byte> and refers to a pointer to a memory location where the ascii byte representation of its characters are stored.

For example, the following snippet will retrieve the ascii byte value of the character 'o' and store it in the value c.

val str:ptr<byte> = "Hello"
val c:byte = str[4]

The characters are also stored with a terminating zero byte after all the characters. This allows the literal strings to be suitably used with external libraries expecting C language strings.

External Unknown Arity Functions

Neither LoStanza nor Stanza supports the definition of functions that take an unknown number of arguments. But there are external libraries containing such functions. The C printf function is the most famous one.

The printf function would be declared like this.

extern printf: (ptr<byte>, ? ...) -> int

Here is an example of calling it from a function called test.

lostanza defn test () -> ref<False> :
   call-c printf("The friendship between %s and %s is valued at over %d.\n",
                 "Timon", "Pumbaa", 9000)
   return false

test()

Compiling and running the above prints out

The friendship between Timon and Pumbaa is valued at over 9000.

External Global Variables

Let us suppose that generate_fib was written differently. Suppose that it does not accept any arguments, and also returns void. Instead it will retrieve its argument from a global variable named FIB_PARAM, and store the result in FIB_PARAM when finished.

int FIB_PARAM;

void generate_fib (void) {
   int a = 0;
   int b = 1;
   while(FIB_PARAM > 0){
      int c = a + b;
      a = b;
      b = c;
      FIB_PARAM = FIB_PARAM - 1;
   }
   FIB_PARAM = b;
}

To call the new generate_fib, our LoStanza call-fib function would need to be able to read and write to the FIB_PARAM variable. Here's how to do that.

extern FIB_PARAM : int
extern generate_fib : () -> int

lostanza defn call-fib (n:ref<Int>) -> ref<Int> :
   FIB_PARAM = n.value
   call-c generate_fib()
   return new Int{FIB_PARAM}

println("fib(10) = %_" % [call-fib(10)])

Compiling and running the above prints out

fib(10) = 89

Function Pointers

Certain C libraries tend to make heavy use of function pointers for implementing callbacks or parameterized behaviour. Let us suppose there is a C function called choose_greeting that when given an integer argument returns one of several possible greeting functions to return. These greeting functions then accept a C string and print out an appropriate message.

void standard_greeting (char* name) {
  printf("Hello %s!\n", name);
}

void chill_greeting (char* name) {
  printf("'Sup %s.\n", name);
}

void excited_greeting (char* name) {
  printf("%c", name[0]);
  for(int i=0; i<5; i++)
    printf("%c", name[1]);
  printf("%s! Heyyyy!\n", name+2);    
}

typedef void (*Greeting)(char* name);
Greeting choose_greeting (int option) {
  switch(option){
  case 1: return &chill_greeting;
  case 2: return &excited_greeting;
  default: return &standard_greeting;
  }
}

The extern declaration for choose_greeting would look like this.

extern choose_greeting: int -> ptr<(ptr<byte> -> int)>

Here's how to decipher that piece by piece. The returned greeting functions all have type

ptr<byte> -> int

The choose_greeting function returns a pointer to a greeting function. So the return type of choose_greeting is

ptr<(ptr<byte> -> int)>

And choose_greeting, itself, requires an integer argument. Thus the full type for choose_greeting is

int -> ptr<(ptr<byte> -> int)>

Here is the LoStanza greet function which takes an integer argument called option and greets Patrick appropriately.

lostanza defn greet (option:ref<Int>) -> ref<False> :
   val greet = call-c choose_greeting(option.value)
   call-c [greet]("Patrick")
   return false

Notice that the value greet has type ptr<(ptr<byte> -> int)>, and thus it needs to be first dereferenced before it can be called.

call-c [greet]("Patrick")

Let's try it out!

println("Option 0")
greet(0)

println("\nOption 1")
greet(1)

println("\nOption 2")
greet(2)

Compiling and running the above prints out

Option 0
Hello Patrick!

Option 1
'Sup Patrick.

Option 2
Paaaaatrick! Heyyyy!

The Address Operator

The greet function in the previous example accepts an integer argument to select the type of greeting, but it only ever greets Patrick. Let's generalize greet to accept whom to greet as well.

We want greet to be callable from Stanza, so the name will be passed in as a String object.

lostanza defn greet (option:ref<Int>, name:ref<String>) -> ref<False> :
   ...

But the greet function requires a ptr<byte> as its argument, and name is a ref<String>. How do we get access to a pointer to the string's characters?

The String type is declared in the core library as

lostanza deftype String :
   length: long
   hash: int
   chars: byte ...

The ellipsis following the byte indicates that the String object ends with a variable number of trailing byte values. We need a pointer to those values to call greet with. To do that we will use the addr operator, which will return the pointer address of a location.

Let's now write our greet function with the addr operator.

lostanza defn greet (option:ref<Int>, name:ref<String>) -> ref<False> :
   val greet = call-c choose_greeting(option.value)
   call-c [greet](addr(name.chars))
   return false

And update our test code to pass in a different name for each type of greeting.

println("Option 0")
greet(0, "Emmy")

println("\nOption 1")
greet(1, "Patrick")

println("\nOption 2")
greet(2, "Luca")

Attempting to compile the above, however, gives us this error.

Cannot retrieve address of unstable location using addr operator.

What does that mean?

Stable and Unstable Locations

Underneath the hood, Stanza uses a precise relocating garbage collector. What this means is that objects are constantly being shuffled around in memory during the garbage collection process. An unstable location is a location whose address is not fixed, such as a field in a Stanza object. In contrast, a stable location is one whose address is fixed, such as a piece of memory allocated using malloc.

The error above is saying that we cannot use the addr operator to retrieve the address of name.chars, which is an unstable location. name is a Stanza string and will be relocated whenever the garbage collector runs, and so the address of name.chars is constantly changing.

However, we are planning to pass the address of name.chars to C and then immediately start executing C code. Additionally, the C function is guaranteed not to hang onto the pointer after it returns. Thus, in this particular case, we know that Stanza's garbage collector will never have a chance to run, and it is safe to retrieve the pointer of name.chars.

To force Stanza to give you the address of an unstable location, Stanza provides you the addr! operator. So let's update our greet function by using the addr! operator this time,

lostanza defn greet (option:ref<Int>, name:ref<String>) -> ref<False> :
   val greet = call-c choose_greeting(option.value)
   call-c [greet](addr!(name.chars))
   return false

and try compiling and running the program again. The program now prints out

Option 0
Hello Emmy!

Option 1
'Sup Patrick.

Option 2
Luuuuuca! Heyyyy!

You should stick to using the addr operator whenever you can, and use the addr! operator only when you're very sure that the object won't be relocated while you're using the pointer.

Calling LoStanza from C

So far we've only considered calling C functions from Stanza, but what if you wanted to call a Stanza function from C? Stanza supports both directions of calling and this section will explain how.

Let us reconsider the generate_fib function again. This time, we will have generate_fib call a Stanza function for each number that is generated. Here is the code for generate_fib.

#include<stdio.h>
#include<stdlib.h>

void number_generated (int x);

void generate_fib (int n) {
   int a = 0;
   int b = 1;
   while(n > 0){
      number_generated(b);
      int c = a + b;
      a = b;
      b = c;
      n = n - 1;
   }
}

Notice that we assume the existence of a function called number_generated that we can call from C.

C will call number_generated using the C calling convention, so we need to be able to define a LoStanza function that is expecting to be called with the C calling convention. The extern keyword will allow us to do that. Our number_generated function will push the generated number to a global vector called FIB_NUMBERS.

val FIB_NUMBERS = Vector<Int>()

extern defn number_generated (n:int) -> int :
   add(FIB_NUMBERS, new Int{n})
   return 0

The implementation of the call-fib function remains as it was before.

extern generate_fib: int -> int

lostanza defn call-fib (n:ref<Int>) -> ref<False> :
   call-c generate_fib(n.value)
   return false

Let's try it out then! Here's our test code.

call-fib(20)
println("Generated Numbers: %_" % [FIB_NUMBERS])

Compiling and running the above prints out

Generated Numbers: [1 1 2 3 5 8 13 21 34 55 89 144 233
                    377 610 987 1597 2584 4181 6765]

Passing Callbacks to C

In the last section, we showed you how to write a LoStanza function that can be called with C. However, C libraries are not typically architected to directly call a named user function. Instead, the user will pass the library a pointer to a callback function that is then later called by the library.

Let's change our generate_fib function so that it no longer directly calls the number_generated function. It will accept instead, as an argument, a pointer to a callback function which it will call.

void generate_fib (int n, void (*number_generated)(int x)) {
   int a = 0;
   int b = 1;
   while(n > 0){
      number_generated(b);
      int c = a + b;
      a = b;
      b = c;
      n = n - 1;
   }
}

We shall keep the LoStanza definition of number_generated the same, but we will need to change the declaration of the generate_fib function, and also pass a pointer to number_generated to the call to generate_fib.

extern generate_fib: (int, ptr<(int -> int)>) -> int

lostanza defn call-fib (n:ref<Int>) -> ref<False> :
   call-c generate_fib(n.value, addr(number_generated))
   return false

Notice the use of the standard addr operator for retrieving the address of the number_generated function.

Compiling and running the above prints out

Generated Numbers: [1 1 2 3 5 8 13 21 34 55 89 144 233
                    377 610 987 1597 2584 4181 6765]

Let's take this time to review everything that this example demonstrates.

Stanza is calling call-fib, which is a function written in LoStanza.
call-fib is calling generate_fib, which is a function written in C.
generate_fib is passed a pointer to the number_generated function which is written in LoStanza.
generate_fib runs and calls number_generated multiple times.
Each time number_generated is called, it creates a Stanza Int object from the argument passed to it by generate_fib, and calls the Stanza function add to push it onto a vector.

This will likely be the most complicated usage of Stanza's foreign function interface you will come across, but it's nice to know that the flexibility is there when you need it.