To kill a boilerplate - a dev diary
Table of contents
Introduction - the magical world of annotations
Ah, annotations. Truly, one of the most bizarre constructs Java has to offer. Not that they themselves are revolutionary as a concept. The way they are handled, though? Now that's insane... not in a good way. If you've done Java before, chances are you found yourself using weird reflection magic you barely understood to get all the classes you annotated at runtime, all while feeling that there has to be a better way to do it, you're just doing it wrong. Alas, your instincts were tricking you: that's actually how you do it. But, enough about reflection. Let's get onto the topic of this article.
Did you know that you can set something called RetentionPolicy on your custom annotations? Now, what does that mean? Well, it means that you can make it so that your annotation does not last all the way to RUNTIME
, but can only be present in the SOURCE
or CLASS
! Isn't that something?
Now, you may understandably be wondering what would even be the point of such a thing, considering that reflection is a runtime-only thing. How would you use those annotations, if they get dropped before your code even runs? Meet reflection handling's far less known brother: the Annotation Processor.
An Annotation Processor is, in its simplest form, an implementation of the Processor interface, loaded up as a service and fed all items annotated with one or more specific annotations. It gets loaded way early on, before compilation, when the rest of the code isn't actually executable yet, and all you can access is its Abstract Syntax Tree representation. Crazy stuff.
The catch? You can't actually edit the existing classes. You can analyze them, and even generate new ones, but touching the existing ones is not allowed. Wait, what? If you're familiar enough with Java, you may have heard of Project Lombok and its crusade against boilerplate code. If an annotation processor can't edit the existing files, how do @Getter
and @Setter
work? For those that don't know, those are two annotations that, when placed on a field, instruct lombok to generate a getter and a setter, respectively, for that field. So, what gives?
Well, the truth is, they are using a trick. Which is really nothing to be ashamed about - it's entirely Java's fault that there isn't a more legitimate way to do it - but this approach may be a turnoff for some. Why do I even care about this fringe topic? Well...
The problem
Remember my previous post? It's quite the rant, but to give you a summary, it's basically about me struggling to create a way to efficiently do ASM patching in Minecraft Forge for 1.13 and above. The final result of my blood and tears are lillero and its loader. Basically, patches are registered as Java Services, implementing the IInjector
interface we created. As a solution, it's great, because it's elegant and uses no "dirty tricks" to accomplish this. But it also has its drawbacks, namely how tedious it is for the programmer.
Let's take the example patch from the README.md
, for example:
;
;
Three lines of actual patching, rest is boilerplate. Having to manually look up SRG names is horrible, manually registering the patch is a hassle, and even if my DescriptorBuilder
makes generating method descriptors easier, it also makes it considerably more verbose.
Despite these considerable drawbacks, we carried on, because we had an idea: annotations to automate this stuff. We figured that we were not the first ones: as usual, fr1kin's ForgeHax predated whatever we were coming up with. And, once again, implemented it in a completely different way. I can't say I went too deep into the code, but it would seem that fr1kin hooked his own processor into Lombok, rather than making it standalone, letting it do the heavy lifting. Clever, but lazy.
His solution did the trick, but it would've meant bringing lombok into the project. Yeah, I know it doesn't actually get compiled in, but I didn't really want to import a whole suite just for a single use. Instead, I went to work on my own solution.
The story
My first idea was to do whatever Lombok was doing, on my own. Couldn't be too hard, right? Wrong.
First off, Lombok is based on the so-called tools.jar
, an internal JVM API that wasn't really meant for public use. And, while there are Javadocs for it, figuring out how they worked wasn't easy at all. Secondly, apparently Lombok goes in very deep, hooking into Javac itself. That wasn't exactly something I was ready to face quite yet, especially considering the lack of online examples, articles and tutorials on the subject. I resolved to come back to it as the very last option, but luckily I didn't have to.
I tried a different approach. Maybe I couldn't edit classes, but I apparently could generate new ones. Why, that's all I need! At first, I was counting on figuring out a way to print a new version of the class and discard the old one, but eventually I figured out that there was no need. Together with a friend, we outlined how my processor could act in theory. The results were pretty interesting.
Remember the example from earlier? Here is how it would look in the new system:
;
;
;
That's a LOT better, isn't it? I'll explain now, but I'll preface by saying that the class being abstract isn't really a requirement, it's just so the method stub doesnt have two empty brackets.
For starters, it's important to note that all those three annotations do not make it to RUNTIME
. Therefore, calling Minecraft.class
in there is perfectly legal, because that reference won't exist anymore when the class is actually loaded. And imports don't actually exist in bytecode, so that won't be a problem either.
Anyway, my objective was to make a processor that, from that, could create a second class called SamplePatchInjector
, in the same package, looking like this:
;
;
The return value for the method name()
is the extrapolated from the name of the old class, the targetClass() is extrapolated from the class given as value for @Patch
. Optionally, @Patch
can also contain a reason, a string that is printed in the console on injection explaining what the patch does.
methodName()
is just the name of the method annotated with @Target
, and the descriptor for methodDesc()
is also extrapolated from it. Finally, the @Injector annotation can be used on any static void method that takes in a ClassNode and a MethodNode, and that's the one that will be called to actually apply the patch.
It's also implied that a service provider (i.e. META-INF/services/ftbsc.lll.IInjector
) will be made for every IInjector class generated this way. Now that I had a solid idea of how I wanted it to act, it was time to figure out the actual implementation.
Writing an annotation processor
The only in-depth guide about generating classes through an annotation processor I could find was a Baeldung article. It's really not great, but it was the basis for my work, so props to them I guess. I implemented the Processor
interface (rather, I extended the AbstractProcessor
class which implemented it), made the service provider (META-INF/services/javax.annotation.processing.Processor
), and sketched our core, the process(Set<? extends TypeElement>, RoundEnvironment)
method:
For the sake of clarity and of my own dignity, we'll pretend like I knew all along that annotations
referred to the annotations themselves, and not to the annotated classes. I totally didn't change that near the end because I'd completely misunderstood.
Now, let's unpack what happens here. The annotations on the class itself say which source version can we work with - we want Java 8, since Minecraft 1.16 is still on that - and which annotation(s) we want to process.
We get every annotated class that is passed to our method, and check if it's of the Patch
type. This is actually useless, since this processor only has one type of annotation, but I figured it wouldn't hurt. Call it future-proofing, I guess. Anyway, what I'm doing is checking that the user annotated everything properly, which isn't a given at all. That's the job of the isValidInjector(TypeElement)
method:
private boolean
It MAY seem complicated, but it really isn't. The TypeMirror are AST representations of the ClassNode
and MethodNode
classes from ObjectWeb's ASM library, which Lillero uses for patching. Basically, we're checking that within the given class there is one method annotated with @Target
, and one method annotated with @Injector
which is not annotated with @Target
, is public static
and has two parameters, which must be ClassNode
and MethodNode
, in this order. I also make it a point to warn the user when an invalid injector is declared, at the cost of making the code look fugly.
Every injector who passes this test is put into validInjectors
, a Set. Well, finally we got them... what now? I figured I'd start from the actual class generation, which seemed like the hardest part. And thus, I added a validInjectors.forEach(this::generateInjector);
...
...aaand I had no clue of what to write in that method.
Class generation
Baledung's guide made it look like a daunting task: they wrote the entire class, indentation included, line by line. That seems like a recipe for insanity, if you ask me. And, as I was looking for more examples, I found JavaPoet. It generates code in a simple and elegant way: just what I needed! It also turned out to be extremely lightweight (ten or so classes), which is a plus, even if it wouldn't get compiled in regardless.
Alright, now we have a simple API for writing. What's next? I suppose the first thing would have to be getting the values from our annotation, right?
private void
There. That easy, huh? Well... not really. While looking for something else, I stumbled by pure coincidence on a pretty good blogpost explaining that there can be problems in extracting a Class
object from your annotation. Which sucks, since that was my design. On the upside, he does provide a neat trick to at least get the fully qualified name, which is really all I need from the class object. Shoutout to him for writing a good article, even if he is completely incapable of properly formatting his code.
private void
Go read the article if you want to know why or how: it'd be pointless to repeat it here.
The next thing probably has to be getting a representation of the two annotated methods. My first approach was to do it by generating two dummy MethodSpec
s (from JavaPoet), but it turned out to be as broken as it was janky. In the end, I settled for the much more sober (and built-in) ExecutableElement
. The following code should be pretty self-explanatory:
private static ExecutableElement
private void
It'll never be null because we already know from isValidInjector(TypeElement)
that those methods exist, so they can't be null. Now, this is as good a time as ever to build the names for our generated classes. I did it like this:
private void
The while
loop ensures that we get to the package even if the patch is an inner class, regardless of how nested it is. It's important to note that packageElement.getQualifiedName()
would return the name of the class, rather than that of the package, for some stupid reason.
Well, that's about all the preparation we could do, right? Time to actually get to using JavaPoet!
private void
Simple enough; JavaPoet takes care of the actual generation, placing of semicolons and all that. $S
is a String placeholder, telling JavaPoet to place quotation marks around the provided text. Now we get to the ugly part. The others aren't going to be that easy, since we need to generate stuff. For the descriptor generation, I had to write a method to handle that, once again:
public static String
public static String
For SRG mapping, I added a utility class to Lillero for the occasion. Explaining how it works would go beyond the scope of this post; all you need to know is that now the method contains a few more things:
private void
We now have all we need to make the remaining specs:
private void
Writing yet another descriptor generator did irk me; at first, I was really trying to generate code which would call my other generator, rather than a String
directly, but I eventually figured that it was useless. I might still move the descriptor generators over to Lillero, but there was basically no way of reusing the old code as far as I could see. Damn AST.
There's just one method left: the injector. Its spec is a bit less straightforward, but again JavaPoet made it very easy:
private void
I again do that weird getTypeElement(CharSequence)
thing to avoid directly referencing ObjectWeb's ASM so I don't have to import them; everything else should be self-explanatory. Oh yeah: $T
is the JavaPoet placeholder for a type, and will also auto-import whatever gets put there.
Now it's time to create the TypeSpec
, the actual class. I kind of figured out that all method specs except for this last one were pretty much identical, so I made a helper method to reduce code redundancy:
private static MethodSpec
private void
There. now all that's left is actually printing it to a file. There's this thing called a Filer
which is meant to aid AnnotationProcessors in generation, so I used that:
private void
The code above should be self-explanatory. Now, back to the original process(Set<? extends TypeElement>, RoundEnvironment)
method, all that's left to do is generate the service provider. I'm going to need the fully qualified names of the newly generated injectors, so I add one last line to the generation class:
private final Set generatedInjectors ;
private void
Generating resources
It's time to make the service provider. By the way, after writing my own, I stumbled on a processor made to specifically handle service provider generation. If that's all you need, you don't have to write anything yourself, and you can find it here. I still need to make my own, since I'm pretty sure that it can't be applied to classes by another processor. Anyway, back to process(Set<? extends TypeElement>, RoundEnvironment)
I add another isEmpty()
check, in case there's some errors I failed to handle. Then, I call a method, which I'm going to paste all at once here, since it's very straightforward:
private void
All done. The only thing I feel like pointing out is the creative (stupid) way resource location is handled. The empty string is the "package name", as if you'd ever need to place a resource in a package, while you need to write subfolders as part of the filename. CLASS_OUTPUT
is the StandardLocation
you're looking for if you want it to actually be compiled in. If you simply need to generate a service provider but don't want it to be compiled in, for whatever reason, SOURCE_OUTPUT
should do the trick.
Gradle magic
Now, aside from a bunch of debugging that I unceremoniously glossed over, pretending I got everything right on the first try, the only thing left is telling Gradle how to use this darn thing. At first, I thought I'd need to make a proper plugin - fr1kin did it! - but as it turns out it's much simpler than that.
As outlined in the relevant Gradle docs (fair warning, Gradle seems to change doc structure far more often than they should, so this link might break), they use something called incremental compilation, which is meant to speed up the process as much as possible. With Gradle, it's possible for annotation processors to become "incremental" as well, and make use of this feature.
There's three types of incremental annotation processors: isolating, aggregating, dynamic. The explanation is all over the place in gradle, so check out this StackOverflow thread for a better one. Since we get all the info we need from the AST structure, isolating
is the type we need. So, we create the resource META-INF/gradle/incremental.annotation.processors
and inside we write:
ftbsc.lll.LilleroProcessor,isolating
That's it. I added it to BoSCoVicino and it worked. Currently, the way to add it to a build.gradle
is
dependencies
which kind of sucks, but hopefully I'll figure out a better way in the future. If I do, I might edit this post. I did! It's really simple. Now all it needs is
dependencies
...which is about as good as it can get - I might even split the annotations (which is what has to be accessible from the project) from the processor itself in the future.
Here's how I did it. As you may know, we host the builds on our own Maven instance. It was as simple as actually declaring the dependency on Maven:
4.0.0
ftbsc.lll
processor
0.2.1
ftbsc
fantabos.co
https://maven.fantabos.co
ftbsc
lll
0.3.0
jar
com.squareup
javapoet
1.13.0
jar
Obviously (although this simple fact somehow eluded me for days), the build.gradle isn't compiled in and therefore its information can't be retrieved by other buildscripts importing the processor. Now I feel kind of stupid, I even asked this on StackOverflow. Oh, well. All is well that ends well.
Conclusions
Honestly, writing this was actually fun. It's just a shame that the documentation was that sparse - I spent far more time looking up stuff than actually writing code.
It's barebones and needs some tweaking, but Lillero-processor is perfectly usable. Considering the API itself consists of three annotations, it's very unlikely that it will change in ways that affect the functioning. You can check it out on our git instance or on GitHub.