Pagefault We write about things, mostly programming related

Declarative OpenGL (sort of) in D

Normally when writing OpenGL code, it can get very cumbersome/error-prone when defining how to pass vertex data to the GPU. As a given example, heres a toy vertex structure we want to tell OpenGL how to handle when it gets passed in as an array of data.

struct Vertex3f2f {
    Vec3f pos;
    Vec2f uv;
}

With this, the following needs to be done to upload the data to the GPU with OpenGL (and it needs to change if the vertex structure changes, where making these changes by hand every time can easily lead to mysterious buggery)

//some excellent array of vertices
Vertex3f2f vertices = [...];

//storage for vao and vbo
GLuint vao, vbo;

glGenVertexArrays(1, &vao);
glBindVertexArray(vao);

//create 1 buffer
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);

// upload to GPU, send size in bytes and pointer to array, 
//  also tell GPU it will never be modified.
glBufferData(
  GL_ARRAY_BUFFER, 
  vertices.length * vertices[0].sizeof, 
  vertices.ptr, 
  GL_STATIC_DRAW
);

glEnableVertexAttribArray(0);
glVertexAttribPointer(
  0, // attrib index
  3, // number of elements (3 floats, Vec3f)
  GL_FLOAT, // element type
  GL_FALSE, // can normalize elements to range [-1, 1] for signed, [0, 1] for unsigned
  vertices[0].sizeof, // stride for given attrib
  cast(const(void)*)null // offset of the first attribute
);

glEnableVertexAttribArray(1);
glVertexAttribPointer(
  1, 
  2, // Vec2f.. 
  GL_FLOAT,
  GL_FALSE, 
  vertices[0].sizeof, 
  cast(const(void)*)vertices[0].pos.sizeof // initial offset is size of pos
);

//... potentially some other code

In the process of enduring this boilerplate I set out to create a way to generate OpenGL code from a “vertex specification”. … Which is just a convoluted way of describing exactly that Vertex3f2f struct thing up there.

Now that we’re past hinting why this might be a desirable thing, lets put it into action and describe the rest of the process!

Prelude to Code Generation

Given our type definition and D’s metaprogramming capabilities, we can actually iterate over all members in the struct we pass at compile-time and filter so we only get POD members.

template PODMembers(T) {

  import std.meta : Filter;
  import std.traits : FieldNameTuple;

  template isFieldPOD(string field) {
    enum isFieldPOD = __traits(isPOD,
      typeof(__traits(getMember, T, field)));
  }

  alias PODMembers = Filter!(isFieldPOD, FieldNameTuple!T);

} //PODMembers

//can be used like
import std.meta : AliasSeq;
alias Types = AliasSeq!(int, float, double, Object);
alias PodTypes = PODMembers!Types;
//should give us (int, float, double)

Now, except for those mysterious __traits__ instrinsics (explained here), it should be fairly straightforward for anyone familiar with filter and map primitives in most FP languages.

FieldNameTuple gives us a tuple of all the field names (as strings) in a given type, which we us as an input to our Filter computation. Filter here expects a predicate of the form:

template SomePredicate(Thing) {
  enum SomePredicate = SomethingWhichReturnsABoolean;
}

We also need to define a function to convert the given primitive member of each vector in the vertex structs to the given OpenGL equivalent, int -> GL_INT for example.

Lets define a template which basically is just a big switch on the primitive types.

template TypeToGLenum(T) {

  import std.format : format;

  static if (is (T == float)) {
      enum TypeToGLenum = GL_FLOAT;
  } else static if (is (T == double)) {
      enum TypeToGLenum = GL_DOUBLE;
  } else static if (is (T == int)) {
      enum TypeToGLenum = GL_INT;
  } else static if (is (T == uint)) {
      enum TypeToGLenum = GL_UNSIGNED_INT;
  } else static if (is (T == short)) {
      enum TypeToGLenum = GL_SHORT;
  } else static if (is (T == ushort)) {
      enum TypeToGLenum = GL_UNSIGNED_SHORT;
  } else static if (is (T == byte)) {
  	enum TypeToGLenum = GL_BYTE;
  } else static if (is (T == ubyte) || is(T == void)) {
        enum TypeToGLenum = GL_UNSIGNED_BYTE;
  } else {
      static assert (0, 
          format("No type conversion found for: %s to GL equivalent", 
                  T.stringof));
  }

} //TypeToGLenum

Actual Code Generation

Given the previous definitions, lets make a simple function which will accept a vao and vbo, as well as an array of vertices and in the process, generate the code to upload the data for the given vertex type to the GPU.

/* templated on VType
  would look something like this in c++:

    template <typename VType>
    void uploadVertices(...etc)

*/
void uploadVertices(VType)(ref GLuint vao, ref GLuint vbo, VType[] vertices) {

  glGenVertexArrays(1, &vao);
  glBindVertexArray(vao);

  glGenBuffers(1, &vbo);
  glBindBuffer(GL_ARRAY_BUFFER, vbo); 

  //assumes GL_STATIC_DRAW here, simplistic example
  glBufferData(GL_ARRAY_BUFFER, 
    vertices.length * vertices[0].sizeof, vertices.ptr, GL_STATIC_DRAW);

  //static foreach, is executed *at compile time and produces 
  // the block of code inside for each iteration*
  foreach (i, m; PODMembers!VType) {

    alias MemberType = typeof(__traits(getMember, VType, m));
    enum MemberOffset = __traits(getMember, VType, m).offsetof;
    //note that types need to provide a _T to access type of members 
    // for example with Vec3f, T_ = float
    alias ElementType = MemberType._T;

    glEnableVertexAttribArray(i);
    glVertexAttribPointer(i
      //computes num of elems by sizeof(Vec3f) / sizeof(Vec3f._T)
      MemberType.sizeof / ElementType.sizeof,
      TypeToGLenum!ElementType, //GL_FLOAT if ElementType is float
      GL_FALSE, //TODO: handle normalization
      vertices[0].sizeof, //stride as mentioned earlier, is size of struct
      cast(const(void)*)MemberOffset //offset for given member in bytes
    );

  }

}

// so if you wanted to upload data for a rectangle
float w = 1.0f;
float h = 1.0f;

Vertex3f2f[6] vertices = [
    Vertex3f2f(Vec3f(0.0f, 0.0f, 0.0f), Vec2f(0.0f, 0.0f)), // top left
    Vertex3f2f(Vec3f(w, 0.0f, 0.0f), Vec2f(1.0f, 0.0f)), // top right
    Vertex3f2f(Vec3f(w, h, 0.0f), Vec2f(1.0f, 1.0f)), // bottom right

    Vertex3f2f(Vec3f(0.0f, 0.0f, 0.0f), Vec2f(0.0f, 0.0f)), // top left
    Vertex3f2f(Vec3f(0.0f, h, 0.0f), Vec2f(0.0f, 1.0f)), // bottom left
    Vertex3f2f(Vec3f(w, h, 0.0f), Vec2f(1.0f, 1.0f)) // bottom right
];

GLuint vao, vbo; //wee
uploadVertices(vao, vbo, vertices[]);

The most mystifying thing up there might be the foreach, when a foreach is operating on a tuple of types it executes at compile time and essentially acts as a sort of rudimentary code generator, producing the block inside the foreach for each iteration. (an example from p0nce’s excellent d-idioms)

So given the uploadVertices definition, it should be 100 % equivalent to what we wrote at the very beginning of the post, except now it works with an arbitrary vertex definition, assuming it defines a _T to get the element type.

Given each different instantiation of the function, another version of this function will be generated, so you might have one for the Vertex3f2f and another for a variant which simply needs to upload a Vertex3f vertex buffer, or… you get the idea.

Now this is a rather simplistic example, which doesn’t handle normalization of attributes, or any kind of instancing related attributes, but already here we have something we can build on! Integrating index buffers might be an interesting idea, but it is out of scope for this quick introduction.

I hope this showed you a little bit of just how useful D’s metaprogramming capabilities can be, and that it wasn’t too big of a mess to read through.

Today I Learned About Locality Sensitive Hashing

I’ve been planning on writing a longer series of post about hash functions to learn more about how they work and what goes into designing good hash functions.

During my research I stumbled upon something called Locality Sensitive Hashing. If you’ve used a hash function before or know something about them, you probably know that changing the input to a hash function by just a little bit produces a wildly different result. Take for example the sha256 hash function with the string: “Hash functions”.

The output of this is: 5a2dba974930153b12b69e46023e73bd09dcdc8216fe2960a991791389f4519c

But if we just change one letter so that we have the string “Hash functjons” we get the output: 457321e76142b2fcb0147ceb8233f4efc7f1d0154c1b1aa48778f905f7de7bc0

The idea behind a locality sensitive hash function is that data that are very similar would hash to similar outputs. This would mean that the example above would with high probability hash to two close hash digests. There are several applications where using hash functions with this property are useful such as malware fingerprinting and similarity between documents.

To read more about this quite interesting subject I would recommend both the wikipedia page: https://en.wikipedia.org/wiki/Locality-sensitive_hashing and this section from Mining of Massive Datasets: http://infolab.stanford.edu/~ullman/mmds/ch3a.pdf

Post 0

Since my esteemed friend profan posted a small intro post I thought that I would do the same.

Welcome to our new shiny blog!

I’m Eric and together with profan (and maybe more people to come in the future) will write in this space. I’m a former computer science science student now working programmer. I’ll be mostly writing about computer science and programming topics interspersed by occasional other topics such as chess and art.

I have some ideas on what I want to do with this blog such as small Today I Learned posts and longer series which I’m thinking about calling This Year I Learned which will be long running posts about a single topic that I want to learn deeper about.

Hopefully it will be useful for some people, at least I will probably be more motivated to learn and share!

Hello, World!

Welcome!

This newly instated programming and general computer science related blog will our little corner of the blogosphere where we post our assorted ramblings and adventures in convoluted solutions.

We all share an interest in computer science and will hopefully produce something interesting along that note.

Subscribe to our atom feed and get all the nonsense right when it comes out!