MMO Fortran

FORTRAN? or Fortran?

isn’t it supposed to be FORTRAN? No, it was renamed to lowercase Fortran, in the 90s to give an easy way to tell the difference between the old ‘structured’ style and the ‘free form’ style language.

Minimal Viable Product

The “Minimal Viable Product” for this is to have a website where a user can create a account, a server which processes multiple client requests, and a client which allows the user to view their player “a cube” and other players cubes moving around in a 3D environment.

If this is the first post you are reading about this project, please refer to my posts on the MMO Project and the testing program design document

For the Fortran implementation, I have decided to go with the minimal viable product only; as this started off as a saturday project and morphed into a multiple saturday project

Implementing the Website

Fortran does not have any native way to connect to the internet. It cannot create sockets of any kind.

This makes it an absolute nightmare to create a web server since you need to import sockets and tcp libraries from C, using the using the iso_c_binding feature.

However, in the spirit of the challenge, I wanted to not have to resort to importing functions from another programming language as much as possible. Later when I create the client I would have to import libraries from C, but for this I could try and use “Fortran Only”.

While reading this old Fortran google groups question from 2009 the group suggested that the user just use netcat and serve the Fortran program as if it were a cgi script.

This was a great idea because instead of needing to mess around with networking and making sure the functions and types imported correctly from C, I could just read and write to standard in/out and to files.

The downside of using netcat is that it uses unidirectional sockets. You would have to create a named pipe and do a bunch of redirecting to get it to work.

mkfifo ncpipe
nc -l 8080 0<ncpipe | ./fortran-micro-httpd '../../common/html/index.html' 1>ncpipe

It just doesn’t seem all that clean to me. Additionally, it refused to stay open after running one message from the web client, so I gave up on that approach.

Something that does work and be a lot cleaner is listen1 from plan9port. This allows for the web browser to connect to the programs stdin, stdout, & stderr, and the program can read and write as a normal command line program.

listen1 'tcp!*!8080' ./fortran-micro-httpd

I next implemented the post handling. In www.f90, it checks to see if the header returning from the web client has data in the body and will process that data, otherwise it will just return the index.html.

I found a library to interface with the sqlite library which will store the username, password, and other information about.

The issue is that fortran does not have a built in cryptographic hash library nor is it in the standard library. So it has been omitted for this implementation, although it should not be in a real world situation.

The implementation does not work well with existing systems since you have to convert from fortran strings to c strings and it returns an integer array of c strings which has to be converted back into fortran strings.

This is a horrible implementation from a security POV, since there is no easy way to sanitize the input, fortran certainly doesn’t do that out of the box.

Also, it turned out that the library simply did not work. Perhaps it was the way I was passing in the fortran strings, but I only ever got garbage back and not a hashed value.

Speaking of strings. At first fortran lulls you into a false sense of security with strings. Fortran has a fantastic way of handling “deferred length arrays”. Which means that you can take an array of unknown size and just keep appending arrays to each other.

For example, this is totally valid in fortran:

character, dimension(:), allocatable :: tape
character, dimension(4) :: magic_module_header = [ achar(0), achar(97), achar(115), achar(109) ]
character, dimension(4) :: module_version = [ achar(1), achar(0), achar(0), achar(0) ]

tape = [magic_module_header]
tape = [tape, module_version]

open(unit=u, file=file_output, access='stream', status='replace', action='write', iostat=ios)
write(u, iostat=ios) tape
close(u, iostat=ios)

deallocate(tape)

Pretty cool for a language with similar performance to C!

Well, its not that simple when it comes to “strings” because there are different ways to structure a “string” in fortran.

! theres the 'suggested fortran 2023 standard way'
! this format is also the way you can use a lot of the built in string functions and the '//' append operator.
character(len=:), allocatable :: string

! this way which is useful since you can use the same interface as a normal 1d array
character, dimension(:), allocatable :: string

! this other other way which only works inside of subroutines/functions
character, intent(in) :: string(:)

! also this older way
character(*) :: string

! and this ancient '77 spec way which wont work in some compiling modes.
CHARACTER string*17

Can you tell the difference between all of them? they’re all doing almost the same thing, but they are slightly different. And some ways seem to work fine until you try to write to output. Or if you use a specific operator on one type, you cant on the other.

This is exactly what happened with the sqliteff library I was trying to use.

Lets take a look at this chunk of fortran code

character, dimension(:), allocatable, intent(in) :: request
integer, intent(in) :: length

character(len=24) :: username
character(len=:), allocatable :: command

get_username: do i = 1, length
    if (request(i) .eq. '=') then
    s_idx = i + 1
    end if

    if (request(i) .eq. '&') then
    e_idx = i - 1
    exit get_username
    end if
end do get_username
username = transfer(request(s_idx:e_idx), username)

command = 'echo ' // trim(adjustl(username))
call execute_command_line(command)

Does this seem like it should work?

It should be that we pair through the request to find the first “username” value.

then use the transfer function which “magically” converts the slice of the request into the username string.

then we call “echo” and run it on the command line. Simple enough right?

Well no, because username is a static size, it keeps it length and will export random junk from other arrays alongside of it.

So we can call trim and adjestl and cut it down to the correct length right? actually no, once again because it is a static size it will generate a bunch of ‘zero length’ characters which will break the sqlite call.

So we have to come up with a completely different way of doing this.

The only consistent way to fix this issue is by using fortran array slices.

character, dimension(:), allocatable, intent(in) :: request
integer, intent(in) :: length

character(len=24) :: username
character(len=:), allocatable :: command

j = 1
get_username: do i = 1, length
    if (request(i) .eq. '=') then
    s_idx = i + 1
    start = .true.
    end if

    if (request(i) .eq. '&') then
    e_idx = i - 1
    start = .false.
    exit get_username
    end if

    if (start) then
    username(j:j) = request(i + 1)
    j = j + 1
    end if
end do get_username
username_len = j-2

command = 'echo ' // username(:username_len)
call execute_command_line(command)

This is terrible code, but it works. We keep track of the length of the string and then we can take a slice out of the total string when we are concatenating the command. So now it works fine.

Now we have fortran string weirdness out of the way we can talk a little about the actual implementation of this.

The request/response headers are pretty easily replicated.

Probably the best feature in the HTTP protocol is actually the Content-Length header, since it allows you to exactly allocate the amount of memory needed to store the body of the request.

There isn’t a whole lot left to implement. Just getting the path to the html file and the database file from the args.

We read the index.html from the file and serve it to the front end on a GET request.

On a POST request it will parse the body and insert the user data into the database.

This is probably the longest I have had to take for the smallest amount of actual features for anything I have created yet.

Also the hex to integer just doesn’t work at all so I had to implement one myself…

The next issue was SQL. There is no native interface with Fortran and SQLite3 (which is the database I am using for this project). The only “native” solution would be to use the execute_command_line subroutine and then read the output from a temp file in order to do sql operations.

So I looked around on the internet for a different solution.

To my surprise I found a project that aims to add a package manager to Fortran called fpm

One of the libraries that was available was a sqlite implementation. So I decided that since I was already a month behind schedule I would stop being so strict with my rule of wanting to use “100% vanilla Fortran” and start using this package manager.

The Fortran package manger is actually pretty fantastic. It has a similar feel to npm if you have experience with that.

Creating a new project is as easy as fpm new project_name and adding libraries can be done by editing the .toml file and running fpm install

The specific library I found for sqlite is fortran-sqlite3.

So I finished up the last few things in the implementation and called it complete.

Implementing the Server

The server implementation is as simple as I could make it.

I found a library called mod_dill from the tcp-client-server demo code from a book on modern fortran. This gave me a simple TCP interface to send and receive “messages”.

Originally I had tried saving formatted fortran data into a string and trying to send that, but when you convert the fortran string to a c-string it often became garbled and unusable when sent through the tcp connection so I stopped using that method.

I found in fpm official registry a json library which would make it much easier to deal with data transfer.

The simplest way to implement the communication between the server and clients would be a series of tcp requests and responses.

• request

  • int :: request_type
    • ping # 0
    • login # 1
    • logout # 2
    • move # 3
  • char(24) :: username
  • double :: x_pos
  • double :: y_pos

• response

  • array :: records
    • char(24) :: username
    • char(24) :: color
    • double :: x_pos
    • double :: y_pos

For each request, the same response is returned with is a list of the logged in users.

The commands that can be sent to the server are logging in, logging out, moving, and the default “ping” which is essentially a noop from the servers pov.

The server uses the same sqlite library as the www implementation

The server loop is:

1. listen for connection
2. read message from client
3. convert c string to fortran string
4. convert the fortran string to a json object
5. run the command from the json object
6. create a json array of the logged in users and send to client.

I wanted to highlight how clean the sqlite and json interface is here.

The devs for the libraries have done a fantastic job of making it in the “fortran style”

step_loop: do
    rc = sqlite3_step(stmt)

    select case (rc)
    case (SQLITE_ROW)
    username = sqlite3_column_text(stmt, 0)
    apperance_r = sqlite3_column_int(stmt, 1)
    apperance_g = sqlite3_column_int(stmt, 2)
    apperance_b = sqlite3_column_int(stmt, 3)
    x_pos = sqlite3_column_double(stmt, 4)
    y_pos = sqlite3_column_double(stmt, 5)

    call json%create_object(user, username)
    call json%add(user, 'apperance_r', apperance_r)
    call json%add(user, 'apperance_g', apperance_g)
    call json%add(user, 'apperance_b', apperance_b)
    call json%add(user, 'x_pos', x_pos)
    call json%add(user, 'y_pos', y_pos)
    call json%add(user, 'username', username)
    call json%add(users, user)
    nullify(user) !cleanup

    case (SQLITE_DONE)
    exit step_loop

    case default
    call db_error(rc, 'sqlite3_step()')
    exit step_loop
    end select
end do step_loop

call json%add(root, users)
call json%serialize(root, str)

rc = msend(connection, f_c_string(str, .true.), &
            transfer(Len_Trim(f_c_string(str, .true.)), 0_c_size_t), -1_c_int64_t)

Implementing the Client

Implementing 3D in Fortran

The client uses the same mod_dill library from the tcp-client-server demo

Also the raylib library. I have generated a fortran interface for the functions necessary for a minimal client. The module is using the new C interop interface implemented in the 2018 spec which makes it fairly easy to create a interface between a C library and Fortran.

I found Raylib bindings for fortran from user xeenypl on github.

Coincidentally, soon after I started on this project I found that user interkosmos has created a new version of the raylib bindings which is much more feature complete than I have.

One thing to note in here is that Fortran does not actually have unsigned values, so you have to use normal integers and use the -fno-range-check flag to tell the compiler that you know what you are doing. Otherwise the colors would not look correct.

For simplicity I have made it so that the login credentials are passed in through the command line instead of a GUI interface.

Fortran has the ability to use the OOP paradigm so I wanted to show that off here with the player module.

module player_mod
   use iso_fortran_env
   use iso_c_binding
   use raylib
   use json_module
   use mod_dill, only: ipaddr, ipaddr_remote, IPADDR_IPV4, mrecv, msend, tcp_connect, &
                       suffix_attach, tcp_close, suffix_detach
   implicit none

   type player
      character(len=:), allocatable :: username
      type(vector3) :: position
      type(color) :: apperance
   contains
      procedure, public :: login
      procedure, public :: logout
      procedure, public :: ping
      procedure, public :: move
      procedure, non_overridable, public :: sync_camera
   end type player

contains

    ! constructor
    type(player) function init_player(username, position, apperance) result(this)
        character(24) :: username
        type(vector3) :: position
        type(color) :: apperance

        this%username = username
        this%position = position
        this%apperance = apperance
    end function

    ! functions here

end module player_mod

The player is able to store their own name, position, and color. There is an interface to allow the client to sync the camera and send the position to the server.

You can also see the json library shine with this implementation. It made it trivial to process the json array that was built and sent from the server.

The only other thing to note is the weird modulo chunk here:

time = get_time()
if (modulo(time, 1.0) .ge. 0.98_c_double) then
    if (player_updated) then
    players = me%move()
    else
    players = me%ping()
    end if
end if

This is because there is not a good way to asynchronously send a call to the server, so I have this hack which uses the current tick time from raylib and run the update/ping function every second or so.

Conclusion

Here is the final product in all its glory!

Check out the code in the fortran section of the repo.

Fortran overall is a 6.6/10.0 on the scale placing it in high D tier.