In other languages: 中文

User:Adil/Modding tutorial

From Official Factorio Wiki
Jump to: navigation, search

Factorio Modding Reference Guide


Prologue

This is a guide to modding. It should cover (eventually) all the following tasks of modding:

  • Adding and modifying items and entities
  • Using scripts to modify how the game behaves
  • Creating a scenario complete with mission

During this guide, a mod demonstrating all the above opportunities will be created.

The guide contains large chapters that mostly contain theory that should hopefully bring an understanding of the modding system of Factorio. Each of those chapters contains a sub-chapter devoted solely to the example mod creation. The cumulative code of all those sub-chapters forms a working mod, you can also download the complete mod from the mod portal using this link. There might be several releases of this mod, all of them are relevant to a certain part of the guide. Starting with 0.1.0, the chapters of this guide will state when next version of the mod should be examined.

Based off FreeER's Step By Step Guide to Modding Factorio. Updated and written by Adil and Gangsir.

The journey ahead

To mod Factorio, you'll definitely need to use Lua. It is a wonderful language, but I won't be able to devote enough time that it deserves in this tutorial. You can finish this tutorial anyway, but later when you embark on your own, you may have to skim at least a short guide (beware obsoletion!) or Lua reference. There's also an almost up-to-date freely-available book written by language creator himself.

Of course, you won't write a mod with a pure Lua alone. When the game executes your scripts, it provides a set of special functions and variables for script to use. Through the use of those functions and variables your code can interact with the game world. If you're trying to implement some new behavior you'll definitely need to familiarize yourself with them. The best place to do so is the official [API help]. Also, the game itself is distributed with copy of that ("doc-HTML") folder, so it may be viewed without an internet connection. That documentation is rather laconic, but complete and up to date. The [DataLifecycle] page of that documentation should also be mentioned, it is like well-hidden, shorter and crispier version of this opus. You can also search through the wiki Modding section, although its relevance is diminishing over time. (You don't need to read up all that just now, but remember those places when the need arises.)

Obviously you will need a text editor, get one with (Lua) syntax highlighting. Examples are: Notepad++, TextPad and Geany.

For the creating of images and icons you may use Photoshop, Paint.net and Gimp.

For testing purposes, it is better to have a separate instance of the game with just the base mod installed. This is easily done by copying a new instance of the non-steam version of the game. Mods and Application directory pages might bring additional insight on that if needed.

The game is always writing to its logs, those are useful when something doesn't work. Default file for the last session is factorio-current.log. Functions like error() and print() output their stuff there. You can also launch the game executable through terminal or command line and then the text output will be duplicated there in real time.

And remember: keep backups! More than one potential mod was lost to freak accidents and miscalculated cleanups.

Changing API

This guide is not written at the moment you are reading it. The game is in development and so the rules of modding change with it. It is possible that not everything in this guide will work. If you get runtime script errors, consult with changelog located in /data, use ctrl-f for any related keyword. If the game stops loading see the errors in the logs. Most likely it's some prototype error, those can be overcome by comparing your prototypes with those defined in the base mod.

Of course there's also forums you can turn to for help.

What is a nature of a mod?

During the start-up process the game scans the /mods (actually, /data and /mods) folder for any sub-folders and looks for an info.json file in it. When it finds that, you have a minimal mod.

After that, it runs script files named:

  • data.lua
  • data-updates.lua
  • data-final-fixes.lua

In that order. The execution of these files from all mods leave the game with the table describing all objects it can have.

When the gameplay itself begins (new game started, save is loaded) the program checks the mod folders for a control.lua file and executes that. The task of the control.lua file is to register functions that will be called upon one or another event happening. By creatively using those, one can make things act like if they are other things. Teleporters, sentient constructs and quests, it all goes here.

The very simple mod

So, by now you have been provided a vague outline of what it means to write a mod, and probably have already found the folder where mods are to be stored.

This chapter will cover the creation of a basic mod: creation of info file, definition of prototypes, adding translations and coding a new mechanic to the game. The examples below use the code from the "First_mod_0.1.0".

The mod we will create here will add a simplistic implementation of teleportation chests. There will be a sender chest that will draw energy from the network and teleport its contents to the nearest receiver chest. This simplistic idea is chosen because of its simplicity, which allows us to outline the main idea of modding without bumping into a pile of complications the real mod would drag behind itself.

The info.json information file

Let's begin with the only mandatory item inside the mod: a file that describes what this mod is. The game evolves, and its format changes somewhat, so it is probably best to peek into the info.json of the most recent "base" mod for guidance. (It is located in factorio/data/base). Or, look at an already updated mod's file.

So, the format may change by the time you get to read this line, but currently, it is as follows:

 {
     "name": "1337mod",
     "version": "0.1.0",
     "title": "My super mod",
     "author": "Me",
         "contact": "superboss@mymail.org",
     "dependencies": [
         "base", "133mod >= 0.1.1",
         "?somemod",
     ],
     "description": "My super mod that adds cool stuff. Mandatory install and do not steel.",
     "factorio_version": "0.14",
     "homepage": "my_site.com"
 }


Quotes, colons, commas and brackets are all important, don't skip them. title, author, contacts, description, and homepage can be anything. A concatenation of name and version with underscore between them should match the name of the folder of your mod (for example: 1337mod_0.1.0). factorio_version is there to break your mod from time to time, note that base mod lacks this line, but yours won't work without it. dependencies is a list of the names of mods that are needed for yours to function, you can simply list a name, or specify a condition for the version of the needed mod. If you put "?" in front of the name, it makes the dependency optional. The mods that you depend on are loaded before yours, and their handlers are also executed first. If two mods are independent, their scripts are executed in alphabetical order.

The info.json for our mod

For the project we're building in this tutorial the following info.json will be used.

 {
 "name":"First_mod",
 "version":"0.1.0",
 "title":"The first mod",
 "author":"Me",
 "description":"This mod is example of how to mod",
 "factorio_version":"0.14",
 "dependencies":["base"]
 }


Data.lua and prototypes

Now we move on to creation of things in Factorio. As mentioned before, anything that is exists in Factorio is first defined as a prototype, a table with specific structure and specific values of its fields. During the game boot, the executable launches a single Lua machine and initializes a bunch of global variables in it, namely: data and util. (And almost all the default Lua variables.) Then, all the data.Lua scripts are run in that machine, then all data-updates.Lua scripts and, finally, data-final-fixes.Lua. Those files are true Lua scripts and can do anything: search n-th prime, implement Dijkstra's algorithms, etc.

But the game is expecting that during their execution modifications and additions to data will be performed. The data variable is a table that will be used by game to setup the Factorio universe. In fact, it contains the function extend(self,prototypes) and a table raw. The former is customary way to add new stuff to the latter. It is actually data.raw that holds the prototypes for the game. (You can view the implementation in the file /factorio/data/core/lualib/dataloader.lua)

Note that prototypes are created during the game boot and any changes to data.lua will only take effect after the game execution is terminated and launched again.

As stated before, the data.raw is a table where everything is stored by its type and then by its name: all the chests are stored in data.raw.container (except the logistic chests, that have their own type), all biters are stored in data.raw.unit (sometimes mods add other units as well). You can get the iron chest prototype, for example, by doing this:

chest = data.raw.container['iron-chest']

The function extend is usually called with semicolon syntax (Lua stuff) and is a table of prototypes written by you. It performs minimal error check and stores prototypes in the corresponding index inside data.raw.

To sum it up, if you want to make something new, you do:

 data:extend({
     {type="item",
      name="megachest",
      ...
     },
     {type="container",
      ...
     },
 })

If you want to change something in already existing stuff, you do:

data.raw["container"]["iron-chest"].max_health=300

What should be put instead of ellipsis in the above is a different question. You won't find a complete answer here. As it stands, there isn't a place to find documentation on prototype definitions, the devs have stated that this will be fixed in the future.

Furthermore, the prototype format also changes between the versions, occasionally significantly. Thus, the best course of actions is to search the item of interest in other mods (including the base) and replicate the found format in your own mod. There will be a chapter in the end of this guide to list some tricks modders have learned about the prototypes, but for now just check the base mod.

An emphasis should be put on the fact that the prototype must have a certain format and certain fields set. You can set additional fields if you want to use them during the prototype creation stage, but any field that is not coded into a particular type definition will be completely discarded by the game when data.raw is parsed, due to how the game interfaces between its C++ components and Lua components. What this means is that you can't just add inventory_size = 80 to an entity of type assembling-machine and have it also double as a storage chest. Such tricks require the use of runtime scripting in control.lua, or may be entirely impossible, depending on the intensity of the change.

The least taxing method of creating a new prototype is outright copying the working one from the base mod and altering the needed fields, whilst leaving the rest be. Among the other Lua files, the core mod provides util.lua, which adds the deepcopy function to table library:

 require "util"
 local new_chest=table.deepcopy(data.raw['container']['iron-chest'])
 new_chest.name="mychest"
 new_chest.inventory_size=30
 new_chest.minable = {mining_time = 1, result = "item-mychest"}, --for this to compile there should be item with name "item-mychest" defined as well
 data:extend{new_chest} --just in case: in lua you can write ({...}) simply as {...}

As a final note on the data generation, it should be pointed out that all the data scripts from all the mods are run by the same Lua-machine, and, thus, share all the global variables. That is, if you define a variable say:

settings={gen_turrets=true, gen_guns=false}

Then all the data scripts executed afterwards will see this variable as well. Normally, this is not much of a problem, after all, by that time your script should already have done its job. But one must ensure that default variables such as table, string or debug remain usable after the script is done. That is, if you absolutely need to do something like this:

string='my super letters'

Then do it like this:

 old_string=string
 string='my super letters'
 --do your business here
 string=old_string

Creation of our mod's chest

For this mod lets make a simple teleport chest. In order to do so, we'll need to write a bit of scripting in control.lua, but for now let's focus on defining prototypes for the mod. We will need an entity to represent the sender end of teleport, an entity to imitate the receiver, items to place and mine them, recipes to create the items, and a tech that will unlock the recipes.

We'll be creating them here as copies of already existing prototypes and altering the relevant fields in the copies. You should probably view the code through a specialized editor with syntax highlighting, it will help to see the comments.

Firstly, some auxiliary stuff. Technology, recipes, items, those things are quite simplistic and boring.

--place the following in data.Lua
 local technology={
     type = "technology",
     name = "logistics-teleportation",
     icon = "__base__/graphics/technology/logistics.png",
     effects ={
         {type = "unlock-recipe",
         recipe = "teleportation-chest"},
         {type = "unlock-recipe",
         recipe = 'teleportation-chest-receiver'},
       },
     prerequisites = {"logistics-3", "automation-3"},
     unit = table.deepcopy(data.raw.technology['logistics-3'].unit),
     order = table.deepcopy(data.raw.technology['logistics-3'].order),
 }
 technology.unit.count=technology.unit.count*3


 local recipe={
     type = "recipe",
     name = "teleportation-chest",
     enabled = false, --this one may be set to TRUE to make recipe available from the start
     ingredients ={
         {'advanced-circuit',5}, {'steel-plate',5}, {'electronic-circuit',5},
     },
     result = "teleportation-chest-item", --note: you can give same name to the tech, recipe, item and entity if you want, but you don't have to
   }

This is so boring that we will do a complete copying whenever we can. We do the copy and then alter the fields that are important to us.

 local receiver_recipe=table.deepcopy(recipe)
 receiver_recipe.name='teleportation-chest-receiver'
 receiver_recipe.result='teleportation-chest-receiver'

 local item=table.deepcopy(data.raw.item['steel-chest'])
 item.name="teleportation-chest-item"
 item.place_result="teleportation-chest-entity"
 item.icons={ --we can make compound icons using this format
   {
     icon = item.icon,
   },
   { --and we can tint them
     icon = "__base__/graphics/icons/iron-plate.png",
     tint = {r= 1, g = 1, b = 0, a = 0.3},
   },
 }


 local receiver_item=table.deepcopy(item)
 receiver_item.name='teleportation-chest-receiver'
 receiver_item.place_result='teleportation-chest-receiver'
 receiver_item.icons[2].tint={r= 0, g = 1, b = 1, a = 0.3}

Definition of entities is a bit more complicated. Entities have quite a lot fields depending on their type and many of them matter. That does not mean that the steal'n'paint approach doesn't work here, though.

The sender will be a chest for which we will write some additional scripts later on.

 local entity=table.deepcopy(data.raw['container']['steel-chest'])
 --quite a bit of fields is going to be changed, let's do that in orderly fashion
 local fields_to_change={
     name="teleportation-chest-entity",
     icons=item.icons,
     minable={hardness = 0.2, mining_time = 0.5, result = "teleportation-chest-item"},
     inventory_size = 5,
 }
 for k,v in pairs(fields_to_change) do
     entity[k]=v
 end
 entity.picture.tint={r= 1, g = 1, b = 0, a = 0.8}

We will also spawn a special unseen entity of ElectricEnergyInterface type for each sender chest. We will use that to implement energy costs for teleportation. Remember, we can't directly make a prototype that will behave as a container and use energy on its own.

 local power_entity=table.deepcopy(data.raw["electric-energy-interface"]["electric-energy-interface"])
 local fields_to_change={
        name="teleportation-chest-power",
        icons=entity.icons,
        minable=false, --Lua stuff: if I'd put <code>nil</code> here, this field would be absent in <code>fields_to_change</code> table but I want to erase this field in <code>power_entity</code> later, hence this hack. (Don't bother much about it)
        working_sound=false,
        selection_box=false,
        collision_box=entity.collision_box, --the power signs are displayed in the middle of this area, so we need it to be the same as our chest (and not 2x2 like the entity we copied)
        picture={
                filename = "__core__/graphics/empty.png",
                priority = "high",
                width = 1,
                height = 1,
                shift = {0,0}
        },
        energy_source ={
       type = "electric",
       buffer_capacity = "1000kJ",
       usage_priority = "secondary-input",
       input_flow_limit = "1000kW",
       output_flow_limit = "0kW"
     },
     energy_production = "0kW",
     energy_usage = "0kW",
     --the game believes all the stuff should be accessible through map editor and needs an order string to sort that
     order='z-z-z',
 }
 for k,v in pairs(fields_to_change) do
        --this logic erases the field if <code>v</code> is <code>false</code>, thankfully I don't have fields that are Boolean <code>false</code> and need to stay that way
        power_entity[k]=v or nil

 end

A receiver will be simply a fancy named chest for our mod. Its prototype will not be at all different from sender, they're both chests after all.

 local receiver=table.deepcopy(entity)
 --for comparison, this one will be done unfancifuly
 receiver.name='teleportation-chest-receiver'
 receiver.minable.result='teleportation-chest-receiver'
 receiver.inventory_size=20
 receiver.icons=receiver_item.icons
 receiver.picture.tint={r= 0, g = 1, b = 1, a = 0.8}


And now the most important thing: we push our prototypes into data.

data:extend{technology,receiver_recipe,recipe,item,receiver_item,entity,receiver,power_entity,}


If you look into the base and some other mods, you may see that those only do a bunch of requires in the data.lua. Any of those required files is capable of accessing global variable data. Using such structured approach is a wise thing to do, but is not absolutely required.

Translation of the mod

Another boring but needed part of the modders' work is translating or making their mods translatable. Any name defined in data lua, is normally not shown in the game itself, instead, the game scans locale files for a string to substitute it with. If it can't find a substitute string, it will print "Unknown key: internal-name" instead.

Every mod can have a locale folder, named with language name abbreviations. For example en is a folder to store localization strings for English. Inside the folder there should be "*.cfg" files. All of them, regardless of name and size are loaded as a dictionary for names. If no translation is available for the language chosen by player, the English translation is used.

Currently, files look similar to following:

 [item-description]
 teleportation-chest-item=Once per second teleports items to nearest receiver.

 [item-name]
 teleportation-chest-item=Teleportation chest

 [recipe-name]
 teleportation-chest=Teleportation chest

Category keywords in square brackets specify the situation where the string should be used, each line starts with the non-localized string, and translation follows after the equation sign. The order of categories is unimportant. Once again, check the base mod for most up to date format.

A few tricks of the trade are:

  • You can pass localized strings in many places that accept ordinary strings by encompassing the string in a table:

game.players[1].print({'entity-name.iron-chest'})

  • A string without a category is allowed, you can use such strings to translate your guis or print messages to players. Such strings are defined at the beginning of the file, before any of the identifiers in square brackets.
  • One underrepresented category is [controls], it's sole usage example can be found by closely studying the core mod, that category is used for translation of key settings menu if you define custom input keys.


The English translation of our mod

For this mod we will make an English translation:

 #this is a comment
 #locale/en/stuff.cfg
 [entity-description]
 teleportation-chest-entity=Once per second teleports items to nearest receiver.

 [entity-name]
 teleportation-chest-receiver=Teleportation chest receiver
 teleportation-chest-entity=Teleportation chest
 teleportation-chest-power=Teleportation chest

 [item-description]
 teleportation-chest-item=Once per second teleports items to nearest receiver.

 [item-name]
 teleportation-chest-item=Teleportation chest
 teleportation-chest-receiver=Teleportation chest receiver

 [recipe-name]
 teleportation-chest=Teleportation chest

 [technology-name]
 logistics-teleportation=Teleportation

Control.lua and runtime scripting

And here we arrive to the most interesting part: lua scripting.

When you see the the factory working, biters running around, stuff happening, you see it because the game engine is performing updates of the in-game state: calculates power consumption, checks where items should be grabbed by inserters, and does myriads of other atomic actions needed for the in-game universe to tick. All those low level calculations are implemented in C++. The modders do not have the power to tamper with that mechanism.

However, during the update calculation the game can detect that a particular operation was notable. For example, determining that a particular unit has its health below zero, or some player clicked an empty ground with a particular item, etc.

When something worthy of note happens, it is called an event. The game packs some relevant information about the event into a lua table and then passes that information to handler functions that were registered to that event by mods. To register those handlers is a prime task of the "control.lua" script file.

These handler functions are a (preferably) brief set of instructions that should do whatever it should quickly and return control to the main loop of the game. Usually, any additional logic you want to introduce with your mod is split over several different handlers. The existence of global variables in lua allows one handler to influence the execution process of other functions.

A more educational example will be given in the sub-chapter below, but a quick sample of handler registration may be:

 --there are many places where you can put handler definition, this is one of them
 local handler=function(event_info)
     game.players[event_info.player_index].print("You have built "..event_info.created_entity.name)
 end

 --this is the registration of the handler to a particular event
 script.on_event(defines.events.on_built_entity,handler)

The function above uses the factorio-API object accessible through the global variable game and other data provided by the game during the event. In this case, it is the unique player_index of a player that has built something and an object representing the entity that was built. The handler function uses that to print the name of the built entity for the player.

The game has quite a lot of special objects bundled with lots of methods the mod can make use of. The official API site is the best place to educate oneself about the available possibilities. It also mentions available global variables and documentation for events. When you embark with your own mod, that page will be your most frequently used.

For now let's talk a bit more about the life of variables in control.lua . During the control.lua execution, you can declare variables to aid your event handlers. For example:


 function on_build(e)
     --this initializes a global variable
     last_built=e.created_entity.name
 end
 function on_mine(e)
     --in this function we will use:
     if last_built then game.players[e.player_index].print("Last thing that was built is "..last_built) end
 end
 script.on_event(defines.events.on_built_entity,on_build)
 script.on_event(defines.events.on_mined_entity,on_mine)

Unlike data.lua, each control.lua is executed by an independent lua machine (in an isolated environment), thus the only constraint to your naming is common sense. However, the example above has one problem: it is inconsistent against the save/load process.

When the game is saved it stores a bunch of data that will allow the simulation to continue from where it was left. To save the state of your lua machine is a difficult task, thus game does not try to save it completely.

Instead, you're provided with a special global variable named global. Its purpose is to be preserved along the savegame. When the savegame is loaded next time, the global is initialized with its contents from the last time.

 --this is a lua global
 a=1
 --this is a Factorio global
 global.b=2

  --you can access these variables in the following way
 print(a) --this prints the variable to std output
 print(global.b)
  --you can't access the factorio-global this way
 print(b)


Generally, any variable that changes its value during runtime you should store in the global table.

One more thing about the control.lua, its execution finishes before most of in-game data is set up or restored. So you should only use the code outside handlers to set up some general data structures, and do not try give items to the player there for example.

If you need to do something when the factorio universe does come to existence (quite frequent need), you register two special handlers: on_init and on_load, try putting the following example in the "control.lua".

  --this output you will always see in stdout
 print('The control.lua start')
 a=1
 print("The global a is",a)
 print("The global b is",b)
 print("The <code>global</code> c is",global.c) --global is always empty at this point
 local initialization=function() --define handler
      --this output will happen when your mod is loaded to the current game for the first time (even if the game itself is not new)
     print"On init is running"
     print("The global a is",a)
     b=2
     print("The global b is",b)
     global.c=3
     print("The <code>global</code> c is",global.c)
     print"On init is done"
 end
 script.on_init(initialization) --register handler
  --you can also define handler during the registration itself:
 script.on_load(function()
      --this output will happen when the savegame is loaded and the mod was present during the save process
     print("On load is running")
     print("The global a is",a)
     print("The global b is",b)
     print("The <code>global</code> c is",global.c)
     print"On load is done"
 end)
 print('The control.lua end') --guess when this output will take place

The handler you register with on_init is called the first time the game is started with your mod enabled. It may be the very beginning of the new game or loading of the savefile. The on_load handler is called during each load after that, this includes the multiplayer, where this handler is executed for the player that connects to the game.

The purpose of the on_init is obvious: if you need to initialize some variable with something from the game wold, you do that here. The on_load event however... You may remember that the game can't store the whole lua state. Some lua data, such as metatables, local variables or closures, simply cannot be saved (serialized) easily. What data can be serialized, and needs to be saved, you store that in the global table.

However the unsaveable data is sometimes needed and is used quite widely in lua programming. So, the on_load event is provided to you, so that you can generate the unsaveable data anew. A simple enough mod doesn't need the on_load handler at all (nor it needs on_init). Also, don't ever try to change the values stored in global during this event. As was noted on_load is run for only one person, so in multiplayer that would lead to people having different values of the same variable, which can cause crashes or desyncs.

Scripting for the tutorial mod

That's enough theory to finally do scripts for our mod. Beware, the code below was written with the intent of being brief and understandable in each separate step. Later we will talk about the improvements of the logic used in the mod, but the optimized version will require clear understanding of the whole picture at once.

So the "design" of the mod is the following: player builds sender and receiver chests, then once per second each sender chest searches for the nearest receiver in a certain range and transfers items to that, the transfer costs some energy, the cost is exacted per transferred item type. The last part is also a simplification for clarity.

This design will be implemented this way: when something is built, the mod checks whether the built entity is either sender or receiver, and stores those in global, then each second senders stored inside global will be checked for items, energy, and then the corresponding receiver will be searched and item transfer will be attempted.

Here are the contents of control.lua file. We begin with some constants definitions. The differences between local and global are nonexistent for the current mod, we will discuss the some time later.

  --control.lua
 local evs=defines.events --defines is always accessible
 MAX_TELEPORTATION_RANGE=100 --some global parameter for the mod
 TELEPORTATION_ENERGY=10000 --this is equivalent to 10kJ I guess.

We do some initialization. We could get away without this but let it be for now.

script.on_init(function() global.senders={} global.receivers={} end)

We then subscribe our handler for building events.

 --you can pass a table with several event-id's to register the same handler to all of those
 script.on_event({evs.on_built_entity,evs.on_robot_built_entity},
     function(e)
          --this is reaction to our entities being built
         local en=e.created_entity
          --first we check if it's our entity
         if en.name=='teleportation-chest-receiver' then
             local f=en.force.name
             global.receivers[f]=global.receivers[f] or {} --this is common table initialization\check trick
             local uid=en.unit_number --many entities have this field, which is guaranteed to be unique
             global.receivers[f][uid]=en --we store the reference to our receivers on per-force basis
         elseif en.name=="teleportation-chest-entity" then
              --yes, with a bit of thought it was possible to avoid code duplication
             local f=en.force.name
             global.senders[f]=global.senders[f] or {}
             local uid=en.unit_number
              --let's spawn our additional power interface
             local p=en.position
             local pow=en.surface.create_entity{name="teleportation-chest-power",position={p.x-0.5,p.y-0.5},force=en.force}
             pow.destructible=false --let this one be untouchable
              --now we'll store both entities for the future use
             global.senders[f][uid]={chest=en, power=pow}
         end
     end
 )

And here we murder the fps. (not too much, honestly) Actually, there are numerous ways of making this even slower, and those don't even include explicitly unnecessary actions. As said before, improving over this simple function will require modifications not only to this handler but rethinking the whole implementation (all 50 lines of code), so let it be this way for now.

  --here we'll make our chests teleport items to nearest receiver from time to time
 script.on_event(evs.on_tick,function(e)
  --this function is called on each game tick(update of the game state)
  --it will murder the game's fps if done badly

     if not (e.tick%60)==0 then return end  --this is common way not to do much work on every tick, we just don't need that frequent updates for our mod
     for force_name,receivers in pairs(global.receivers) do --for each force that has built receivers
         if global.senders[force_name] then  --we check if it also has senders
             for uid,tab in pairs(global.senders[force_name]) do --and run updates on those
                 local chest=tab.chest
                 if not chest.valid then --something broke the chest is no more
                     if tab.power.valid then tab.power.destroy() end --remove its power drain from the game as well
                     global.senders[force_name][uid]=nil --forget about it
                 else
                      --let's find where to send items, let's put that logic in a separate function for example
                     local nearest,distance =find_nearest(receivers,chest.position,MAX_TELEPORTATION_RANGE)
                     if nearest then
                      --lets try to transfer items
                         local receiver_inventory=nearest.get_inventory(defines.inventory.chest)
                         local sender_inventory=chest.get_inventory(defines.inventory.chest)
                         local items=sender_inventory.get_contents()
                         local p=tab.power
                         if p.energy> TELEPORTATION_ENERGY then --very primitive logic for tutorial
                             for item_name,count in pairs(items) do
                                 local inserted = receiver_inventory.insert{name=item_name,count=count}
                                 if inserted > 0 then sender_inventory.remove{name=item_name, count=inserted} end
                                 p.energy=p.energy-TELEPORTATION_ENERGY
                             end
                         end
                     end
                 end
             end
         end
     end
 end)

 find_nearest=function(entities,position,max_distance)
      --this is auxiliary function used in the above handler
      --it is stored in global variable, thus it will be seen during the <code>on_tick</code> handler, despite the fact that it is defined after the handler. The important thing is that the handler is definitely called later than this function is created

     local distance=max_distance^2
     local nearest
     for k,e in pairs(entities) do
         if not e.valid then
              --in the current disposition, this is the only place to check for receiver validity
             entities[k]=nil
         else
              --this part is pretty standard linear search algorithm
             local p=e.position
             local d=(p.x-position.x)^2+(p.y-position.y)^2
             if d < distance then
                 nearest=e
                 distance=d
             end
         end
     end
      --if we have found something, we return it
     if nearest then return nearest, math.sqrt(distance) end
 end


This concludes the "First_mod_0.1.0"!

The remaining sections deal with creation of scenarios, some tricks of trade and updating the mods. At this point you probably have enough knowledge to, with a bit of luck, make your own functional mod. You may read on, if that is your thing, or you may try to mod and run back here if something fails.

Additional information

There is more to factorio modding than has been already said. In this part of the guide we'll talk about creation of scenarios, there probably will be a few more things to tell about lua coding and performance, and finally there will be talk about updating your mod.

A tale of tales - Scenarios and Campaigns

The examples of this chapter are from the "First_mod_0.2.0". Basically, a scenario is a set of completely optional parameters to start a game. When player clicks "custom scenario" button in the menu, the game searches the folders named "scenarios", it looks for them in the "mods" and "data", it also checks the game directory itself. Any folder inside "scenarios" is a scenario.

Thus, a trivial example of a scenario is an empty folder. If you start such a scenario ("First_mod/trivial_example" ,for ex.), the map generator settings window is brought up and after that the game starts. The only difference of such scenario from the "freeplay" is that it won't pester you about rockets without satellite (and it won't tell you that you've won and you'll start without items). That functionality (both the pestering and starting items) is provided by scripting.

Updated my journal

A scenario is allowed to contain a "control.lua" script file. Whatever a mod script can do, scenario's script can do as well. The difference between them is that mod script file is executed at every load when a mod is active, and scenario script is executed when the scenario is loaded. The scenario script is also, copied into control lua, so you need to start a new game to see the changes you've made. (or use migration, we'll talk about it later). A final note is that the script control.lua and the mod control.lua are executed in the independent environments thus both of them have global, these are different globals.

Thus it is entirely possible to move mod's script into a scenario and have the mod features only work in that particular scenario, but the conditions under which that would be reasonable are exotic at best.

A conventional use of the scenario script is to add some goal or a final condition for the game. A simple example of that can be seen in the "First_mod\win_by_chest" scenario, which has the following script:

script.on_event({defines.events.on_built_entity,defines.events.on_robot_built_entity,}, function(e)
    if e.created_entity.name=='wooden-chest' then
        game.set_game_state{game_finished=true, player_won=true, can_continue=true}
    end
end)

Something not as silly can be observed in the freeplay scenario, the point however is: the scenario script uses all the same methods as ordinary script but strives for a different goal.

If you check the base mod campaigns (in /data/base/campaigns) you will notice, that their script files are nothing like above. The reason is that they use a special library located in /data/core/lualib/story.lua . That library though provides no opportunities above the ones already available, it is just a way of structuring happenings in the scenario. You may spend your time trying to understand how to use it by looking at the source and its usage in the vanilla files, or you may design your own, or in case of a small scenario, you may just get away with directly scripting everything.

I need a turtle and a bunch of elephants

Sandbox and freeplay use map generator, but it is of course possible to create a scenario with predefined map in it. If there is a "blueprint.dat" file in the scenario folder then that one is used instead of randomly generated map. That file is the one you can generate with the map editor tool, or map converter tool. The editor is available in the main menu of the game, the converter is semi-hidden feature of the executable.

The converter is a tool to convert an existing savegame into a blueprint for scenario. Basically, you just play the game, then save it, then use command line to launch factorio binary with this command:

factorio -s save_name_here

And then you just hope for the best. Sometimes the converter succeeds and creates a new scenario named after the savegame inside the "scenarios" folder of the game.

That same folder is the place to look for the saved scenarios that you've created in the editor, by the way.

The editor is not much different from many other editing tools you might have encountered already. Except for weird keybindings, check the "editor" tab in the "controls" menu to get understanding of what's going on there.

Let us skim over some worthwhile bits of information.

There are several edit categories: tiles, resources, items, entities, spawn. When a particular category is selected, you can edit objects that fall into that category.

Depending on category, there may be brush or spray tools, they have a couple of parameter that work like they would for photoshop tool of the same name. The left mouse button places, the right button erases, if you have some entity in your cursor (the one you've been spraying) only entities of that type are erased, if you free the cursor (using "q" button) then you can erase anything in the category. When placing tiles, you cannot erase, only paint other tiles over.

In addition to brush and spray there also is a cursor tool. When editing resources, it acts as a brush of size one. When entity category is selected, right click places the entity. Removal is done by "x" button. Right click on entity when the cursor is empty brings up entity GUI, the same as you see in-game, left click changes entity sprite (used for trees). The "t" button opens up flag editor, you can make entity unminable there, or indestructible, or change its force (one by one).

One additional feature of the entity editor is the "tag" field. You may give an entity some unique tag and then in control.lua of the scenario, you can use get_entity_by_tag(tag) to obtain a reference to this object.

Currently the editor does not have copy\paste, nor it allows the copying of settings like you can do in the game.

Editor does not support working with additional forces, you may only change the spawn point of the "player" force.

You can also use scripting for manually shaping the map for your needs. The "team-production" scenario does that and there is a primitive construction library in data/core/lualib/builder.lua .

Can't trade a quest item

A scenario is scanned way after the data stage, so you can not add new prototypes (entities, items anything) in the scenario itself. But if you need, you may define them in the encompassing mod's data.lua. They will then be defined no matter what scenario is loaded, so you'd better ensure they don't appear where they should not. If it is technology or recipe, you should use script to set its enabled field to false. If it is an entity, tile. or resource then you'd have to make sure they're not autoplaced by map generator.

For example, we could do that with the following trick:

 --First_mod/RatRace/control.lua
remote.add_interface("RatRaceScenario",{})

This one features the usage of remote object, it will be discussed later. In a short, it is used to transmit information between the independent lua states. In the current example it carries the information by the very facts of its existence.


hide_tech_and_recipe=function()
    if not remote.interfaces["RatRaceScenario"] then
        for _, f in pairs(game.forces) do --technologies and recipes are stored on per force basis
            for _,t_name in pairs(dummy_tech) do
                if f.technologies[t_name] and f.technologies[t_name].valid then
                    f.technologies[t_name].enabled=false
                end
            end
            for _,r_name in pairs(dummy_recipe) do
                local r=f.recipes[r_name]
                if r and r.valid then
                    r.enabled=false
                end
            end
        end
    end
end
hide_entities=function(event)
    local surface =event and event.surface
    local area= event and event.area
    if not remote.interfaces["RatRaceScenario"] then
        if surface then
            hide_ents(surface,area)
        else
            for _,surf in pairs(game.surfaces) do
                hide_ents(surf)
            end
        end
    end
end
hide_ents=function(surface,area) --this function may be rather slow
    for _, ent in pairs(surface.find_entities_filtered{name='dummy-resource-1',area=area}) do
        ent.destroy()
    end
    for _, ent in pairs(surface.find_entities_filtered{name='dummy-resource-2',area=area}) do
        ent.destroy()
    end
end

And then we stick those two functions into event handlers:

 --First_mod/control.lua
script.on_init(function()
    hide_tech_and_recipe()
     --do whatever else the mod is doing
end)
 --we'd need to track a few other events:
script.on_event(defines.events.on_force_created,hide_tech_and_recipe)
script.on_event(defines.events.on_chunk_generated, hide_entities )


The data and control scripts of the "First_mod_0.2.0" have been updated to define these useless prototypes, feel free to comment out calls to the hide_ functions and see how freeplay changes.

Ok, good. \ No, bad idea. \ I should go

The scenario is allowed to have own "locale" folder. Since scenario can't define game objects, this folder contains only translations of whatever messages you'd want to display in the scenario.


#First_mod/RatRace/locale/en/my_texts.cfg
start-message = No time to explain, RUN!
lose-message = Too slow.
win-message-1 = Yay!
win-message-2 = has reached the finish!


Getting things to work

The map editing is not the strongest side of the factorio, but we have the power of lua to compensate for that. One final example of a scenario will be the "First_mod\RatRace". In that scenario player will have to outrun other participants and reach the end of the obstacle course.

Late addendum: Actually, it turned out that default AI is incapable of guiding units through the course even despite the fact that the route is never completely blocked. Nevertheless, that does not reduce the educational value of the script itself. If you want some competition, try running this one with friends.

The "blueprint.dat" file of this scenario had an eventful life: it was first created in the editor, where the ground was painted. Then I've copied control.lua from the sandbox mode and used that to spawn infrastructure around and set a few circuit networks. I've also used console to set all the stuff unminable:

/c for _,e in pairs(game.player.surface.find_entities_filtered{force='player'}) do e.minable=false e.destructible=false e.operable=false end


Then I've saved the game and fed it to the converter. Then in editor again, a few combinators were marked with tags. Then the "blueprint.dat" was placed in the "First_mod\RatRace".

The accompanying script is given below.

 --First_mod/RatRace/control.lua
remote.add_interface("RatRaceScenario",{})
script.on_init( function()
     --here's the subtlety: since the scenario is a converted savegame, the player[1] is already created in it (and is god-controller type)
     --so we'll have to restore his character
    if game.players[1] then
        game.players[1].character=game.surfaces.nauvis.create_entity{name='player',force='player',position={x=-12,y=500}}
    end
     --we will also spawn some competitors here:
     --edit: actually, nevermind, we do spawn them, but they're no competition when controlled by vanilla pathfinder
    local spawn=function(name,x)
        return game.surfaces.nauvis.create_entity{name=name,force='player',position={x=x,y=500}}
    end
    global.runners={
        spawn('behemoth-spitter',-5),
        spawn('small-biter',15),
    }
    global.backup={}
     --note: this only called once when server starts, so every change here should be stored either in global or in game state itself
     --for example, the following lines will cause desyncs:
     --script.on_event(defines.events.on_tick,step_one)
     --my_constant=42
    global.stage="step_one"
    script.on_event(defines.events.on_tick,step_one)
 --to begin the scenario and to end it, we will only check positions of a few entities on each tick
 --there will be only one event we're interested in:
 --normally, this code would be in the:
 --script.on_event(defines.events.on_player_created, function()
end)
script.on_event(defines.events.on_player_created, function(e)
 --this is in case you want to try this out in the network
    game.players[e.player_index].teleport{x=-12,y=500}
end)
script.on_load(function() --this one is run whenever new player connects and should prevent desyncs
    if global.stage=='step_one' then
        script.on_event(defines.events.on_tick,step_one)
    elseif global.stage=='step_two' then
        script.on_event(defines.events.on_tick,step_two)
    elseif global.stage=='win' then
        script.on_event(defines.events.on_tick,win_sequence)
    elseif global.stage=='lose' then
        script.on_event(defines.events.on_tick,lose_sequence)
    end
end)
step_one= function()
    local t=global.t
    t= t and t-1 or 300 --initial timer (5 seconds)
    global.t=t
    if t<0 then
        for _,p in pairs(game.players) do
            p.print({"start-message"})
        end
        local c =game.get_entity_by_tag"combinator"
        c.die()  -- open the gates locked by circuit at the start
         --send runners to the finish line
        group_check()
        global.stage="step_two"
        script.on_event(defines.events.on_tick,function() step_two()  end)
    end
end
function step_two()
     --lets check who reaches the finish line (we know its position)
    for _, p in pairs(game.players) do
        if p.position.y < -521 then
            for _,e in pairs(global.runners) do if e.valid then e.die() end end
            global.winner=p.name
            win_function()
            return
        end
    end
     --this is the check for other runners reaching the finish line, it is complicated because it struggles against the game engine
    if group_check() then lose_function() end
end
function group_check()
     --factorio is not well set for being rts, hence these complications when working with units
    if not ( global.group and global.group.valid ) then
        global.group=game.surfaces.nauvis.create_unit_group{position=global.backup.group or {0,0},force='player'}
        for _,r in pairs(global.runners) do global.group.add_member(r)  end
    end
    for i,r in pairs(global.runners) do
        if r.valid  then
            if r.position.y<-521 then return true end   --this is the runner reached finish line
            global.backup[i]={name=r.name,position=r.position} --this is because game pathfinder solves its problems by deleting problematic units and groups
        end
        if not r.valid then  --pathfinder has destroyed our unit quietly
            local b=global.backup[i]
            local e=game.surfaces.nauvis.create_entity{name=b.name,position=b.position,force='player'}
            global.group.add_member(e)
            global.runners[i]=e
        end
    end
    global.group.set_command{
        type=defines.command.go_to_location,
        destination={x=0,y=-525},
        distraction=defines.distraction.none}
    global.backup.group=global.group.position
end
win_sequence= function()
    global.t=global.t-1
    if global.t > 0 then
         --do nothing
    else
         --that's enough
        game.set_game_state{game_finished=true, player_won=true, can_continue=false}
    end
end
 --if player won, we set up on_tick handler that will congratulate him for some time
win_function=function()
    global.t=600
    for _,p in pairs(game.players) do
        p.print{"",{"win-message-1"}," ", global.winner, " ", {"win-message-2"}}
    end
    global.stage="win"
    script.on_event(defines.events.on_tick,win_sequence)
end
 --if player lost, we set up on_tick handler that will mock his failure
lose_sequence=function()
    global.t=global.t-1
    if global.t > 0 then
         --do nothing
    else
         --that's enough
        game.set_game_state{game_finished=true, player_won=false, can_continue=false}
    end
end
lose_function=function()
    global.t=600
    for _,p in pairs(game.players) do
        p.print({"lose-message"})
    end
    global.stage="lose"
    script.on_event(defines.events.on_tick,lose_sequence)
end


Tutorial, Module 1, Interlude, Module 2, Afterword...

You may also, add campaigns with your mod. Just make a folder named "campaigns", create a subfolder with a campaign name, copy a "description.json" and image.png file from the "demo" campaign and tweak it to your needs.


{
  "starting-level": "l1",
  "order" : "z",
  "difficulties": [ "easy" ]
}


Campaign also has its own locale folder, there translations of campaign name and description are given:

#First_mod_0.2.0/campaigns/Tutorial/locale/en/lalala.cfg
name=Tutorial campaign
description=Just an example of how to slap together a campaign


The difference of campaign mission from standalone scenario is that when you finish the current mission, you can specify which mission to load next:

set_game_state{game_finished=true, player_won=true, next_level="l2a"}

The next_level is a name of scenario folder within campaign.

You can also define an epilogue with image or without:

game.players[1].set_ending_screen_data({"endText"}, "image.png")


The "First_mod_0.2.0" provides and example of campaign. I've cut all the corners here and it's just a bunch of "win_by_chest" variations. I'll list scripts here side to side so that you may see what I did there:

 --First_mod_0.2.0/campaigns/Tutorial/l1/control.lua
script.on_event(defines.events.on_built_entity, function(e)
    game.players[e.player_index].print('Build either wooden or iron chest')
    if e.created_entity.name=='wooden-chest' then
        game.set_game_state{game_finished=true, player_won=true, next_level='l2a'}
    elseif e.created_entity.name=='iron-chest' then
        game.set_game_state{game_finished=true, player_won=true, next_level='l2b'}
    end
end)


 --First_mod_0.2.0/campaigns/Tutorial/l2a/control.lua
script.on_event(defines.events.on_built_entity, function(e)
    game.players[e.player_index].print('Build iron chest')
    if e.created_entity.name=='iron-chest' then
        game.players[1].set_ending_screen_data({"endText"})
        game.set_game_state{game_finished=true, player_won=true}
    end
end)


 --First_mod_0.2.0/campaigns/Tutorial/l2a/locale/en/epi.cfg
endText=Well, that was fast. You see no image


 --First_mod_0.2.0/campaigns/Tutorial/l2b/control.lua
script.on_event(defines.events.on_built_entity, function(e)
    game.players[e.player_index].print('Build either wooden chest')
    if e.created_entity.name=='wooden-chest' then
        game.players[1].set_ending_screen_data({"endText"}, "image.png")
        game.set_game_state{game_finished=true, player_won=true}
    end
end)


 --First_mod_0.2.0/campaigns/Tutorial/l2b/locale/en/epi.cfg
endText=Well, that was fast. You see an image


The available levels of the campaign are listed in "player_data.json" file, so you may list your campaign missions there and then you'll be able to select those, when you start a campaign. May be used for testing. Also, if nonexistent mission is listed, that is treated like it is a scenario without script.

Lua, Factorio and You - Assorted bits of knowledge

Previously we have discussed how to use control lua. The examples provided there were littered with the details and tricks that relate not so much to control.lua scripting but to usage of the embedded lua in general. In this chapter some additional information on doing Lua inside Factorio will be provided.

setmetatable(this_guide, {__index=Programming_In_Lua})

Once again, there are more well-written guides, tutorials and references on Lua than this guide will ever become. But I felt the need to put emphasis on some topics.

require "clothes boots and motorcycle"

Before everything else: in lua you can call a function with an explicit string argument via the following syntax:

foo('mytext')
 --or
foo 'mytext'

Looks familiar, right? In lua you can split code to a several files and then load the needed code by using require function. The require function reads the code from the file and executes it as a function in a global environment without local variables that may have been defined before function call.

That is: local variables are seen only in the file that defines them, global variables defined in different files are accessible everywhere after their creation. For example:

 --file1
local a=3
b=4
f=require "file2"
c=5
print(c)  --5
print(d)  --6
print(e)  --nil
print(f)  --8


 --file2
print(a)  --nil
print(b) --4
print(c)  --nil
d=6
local e=7
return 8


Additionally, if there's return statement in the file, then require returns that value to the caller. The require has some concealed logic to prevent multiple loading of the same files and that's all special about it.

You can access global, game and global other variables in the files required by control.lua, and define any number of additional global variables. Keep in mind though, that's one way to make a code unreadable. Conventional ways are either creating a single global variable which name corresponds to the filename, or not declaring globals at all and passing everything needed through the return.

As a final note: require can only be called during the mod load process, that is, you can call it in the file but not in the handlers you've registered:


 --example_control.lua
 --this is correct
require "some_file"
 --these are incorrect
script.on_event(defines.events.on_entity_built,function() require "other_file" end)
script.on_load(function() require "even_more_files" end)


Whispers of the world - Debug overlay

By default buttons F4-F7 toggle various debug overlays. The information there is pretty much unobtainable otherwise. Let's leave it up to yourself to familiarize with every other item in those overlay (plus no one knows for sure what do they mean). But most importantly, those overlays provide a mod script execution times (time used) given in milliseconds.

The game runs 60 updates per second, that means, that a single update should preferably take less than ~17 milliseconds, lest the fps drops. Your mods always contribute to the update time (entity update isn't cheap, you know), but if you use scripting, they do it directly and that can be seen in the overlay. You should probably try to reduce that time when you can.

Gotta go fast - Notes on optimization

Sometimes "How do make my mod performance better?" question starts disturbing modder's sleep. The recommendations on writing performing code are rather general, actually. You may consult a brief article [1] by prof. Ierusalimschy for (mostly) lua-specific tricks based on the underlying structure of the language. There aren't many, and frankly, those are only noticeable in the applications of much larger scale. Apart from that the recommendations would be to review the code logic itself and validate that everything it does is indeed required.

Considering the factorio, it is worth of note, that you're writing event handlers here and most events don't really happen all that often. Thus handlers of the most events have little to no impact on the game update frequency. The effect of any optimization will be meager for most events save for on_tick and, maybe, on_entity_die. That is, assuming you don't intentionally do something plain wrong in the script.

And if some function does take noticeable time to finish, then the best bet is it calling some slow function from the API. So, the solution will be once again reviewing the logic and culling unnecessary operations.

An example of such unnecessary operation might be redundant calls to game search functions, those are done in C++, they are fast, but the tasks they fulfill are major as well. Here we are talking about using surface.find_entit* functions, whenever you call those, ask yourself a question "Couldn't I just reuse the previously found reference?".

Actually, reference reuse is universally good. Whenever you receive an ingame object reference (an ingame object) that you know you'll be doing stuff again with, consider storing it. Any object that comes with .valid field (consult API documentation) is eligible for storage.

You probably won't cut any notable time by evading a single index operation:

local player = game.player[1]
global.data = { player = player, force = player.force}
script.on_event(defines.events.on_tick,function()
    local d=global.data
     --the following in not much faster
    d.force.reset_technologies()
     --than this
    d.player.force.reset_technologies()
     --also, the main time hog here is reset_technologies() ;)
end)

But, if used to evade certain retrieval operations like get_control_behavior this trick may be useful.

Trading memory for computation speed is widespread method of acceleration of execution. In the use case above you actually are not trading much. So, unless you start generating your own tables in huge numbers, you won't cause factorio memory usage to go up notably.

We'll get an example in the chapter that deals with the updating the mod.

Perils of coding - Problems, subtle and not

It is easy to create a bug. Most of the time, to fix it, you just need to take another thought on what did you just write. Yet sometimes the error stems from the peculiarities of the underlying system. Some of such cases are listed here:

Saving functions

Storing the execution state of program is not easy. You can easily store the data, your program generates, but saving the state of functions is complicated. In lua functions are data, but saving their state if they have one is difficult. Default distribution of lua does not provide that functionality. And neither does Factorio. So, you can store functions but you cannot save closures. That is a function you're storing can only reference global variables outside its scope. For example if you enter the following lines in terminal (prepending then with "/c "):

a=3
global.f1=function() print(a) end
local b={4}; global.f2=function() print(b[1]) end; global.f3=function() b[1]=5 end

by calling them in the following order:

global.f1()
global.f2()
global.f3()
global.f2()

You'll get:

3
4
5

But if you save and load game, the following commands:

global.f1()
a=2
global.f1()

will yield:

nil
2

The function f1 correctly references global variable (both of them, if you can see the second one) and prints value of a global variable with same name. But if you try to call function f2 however, you'll get runtime error stating that _ENV upvalue is nil.

Tables as keys

In lua the following code is valid:

a={}
global.b={}
global.b[a]=4
global.b[global.b]={}

But when saving\loading is considered, new troubles arise. Some table contents may cause your savegame not to load, and raise get error during running deserialization. You may attempt numerous trivial tests and all of them will work, but that does not guarantee that the system will not stall when used in a real use case.

Object used to be valid

This problem is not as fundamental (well, it is fundamental on the different level) but nevertheless it arises frequently and is ought to be mentioned somewhere. Throughout handling of game events your code receives a whole lot of game-API objects (references to game-API objects). Most of the time they are fresh and safe to use. However, it is always good to check their .valid field before using them. Especially so if you are using previously stored references. As you may notice in the "RatRace" scenario, game engine quietly erases units and unit_groups like no one's business. And some other mods may just call a .destroy() method on something you need in your script.


Community is Magic

You don't have to do everything alone. (Nah, just kidding). But the fact is: some of the problems you'll encounter have been met by someone else before, and if you're lucky, your predecessor might leave something, that may help you as well.

Wisdom of ancients

Below you may find some more specific pieces of information onto some specific problems.

Like rain drops - Autoplace controls

You can make an entity to be placed by map generator. For that, you need to define '.autoplace' field inside the prototype. 'autoplace' is a table with a bunch of fields despite fields being named with quite mundane words, it is mighty difficult to guess what will you get with a particular set of parameters. The links below sum up what is know on noise layers and automatic placement. Types/AutoplaceSpecification https://forums.factorio.com/viewtopic.php?t=2871&p=21497

It is dangerous to mod alone, take this

Here, in order of their encounter by the author, are listed various pieces of work by community, that you can use to save yourself some work. This list is surely not a complete catalog of factorio custom libraries, you may find something neat and not listed if you search forums and mod portal. Also, here are only listed those items that are explicitly named as utility library or function, however any mod is a collection of useful functions itself. Release licenses of most of them allow to reuse their code, so give a credit somewhere and you're good to go, as long as you're not simply copying the mod with a few renames and number changes.

  • Factorio Std Lib One lib to serve them all This one is an ambitious project to provide all things possible. From time to time someone altruistic drags something of use into it, so in a sense (and in a limit of an infinite time passed) the lib obsoletes this particular list. You are not obliged to use it, however. (I personally find it faster to redo things my way than guessing the logic of the library.) But it is there for taking if you find something of interest to you.
  • Cantor pairing This link contains functions you might use to cache entity by its position. The described trick seem faster than naive concatenation of surface name and position numbers.
  • GUI lib The library designed to simplify creation of complicated guis. Comes with example.

Faster, Stronger, Better... Updating the mod

It is not uncommon that you might want to change something in your mod after you've released it. But now people have already run your mod with their game and it is likely the mod did affect the game state in some way. So the question is how to handle that state. Surely, the player can always delete the mod and let the game engine do the cleanup and then install new version of the mod. But the desirable way is to provide a smooth seamless transition.

Some changes in mod do not need any additional thought, for example if you change some function in a script or change, say, hitpoints in entity prototype. The game regenerates that data completely between launches.

With other changes you will not get away that easy:

  1. The global table of the mod is carried over from older version so if your new script expects it to be set up differently than before, you will have to perform the conversion.
  2. Technologies and recipes are saved for each force. If any mod version number changes (including base), these prototypes are reloaded, but the game does nothing to update effects of researched technologies.
  3. Entity type as defined in data.lua is a part of its identity, if you simply change a type of some prototype, say from type='container' to type="logistic-container", then the game will consider that already built entities must be simply deleted and only those built afterwards will be of the new type.

To solve those problems the game provides migration functionality. The official description is provided here. Please read the linked material before moving on. To wrap it up: you create a folder named 'migrations' and put special files there. The filename is arbitrary but should guarantee the correct ordering when migrations are read.

According to the link, "*.json" migration files are there to solve the problem 3 and "*.lua" files - to solve the problem 2. The official documentation provides enough information on these migrations, we will not discuss it further here.

However, there's still a problem 1. "*.lua" migrations do not have the access to the global, so something else is needed in order to solve that. on_configuration_changed() is just the tool for the task. This is the special event handler like on_load and on_init, it is run whenever any mod is added or updated or removed. It has full access to the game state and to the global of the control.lua. Thus it is actually powerful enough to replace the migration files, however for clarity of the code, you should not actually do that. Since it is run even if some other mod is changed, it is also capable of handling situations, when you, for example, use programmatically generate prototypes based on those already present and need to keep track of those during the control lua.

Sins of the paste

Now let's fix some things in the tutorial mod. Get First_mod.0.3.0 version from the portal. In comparison with version 0.2.0, which was used in the scenario examples, this one has those dummy scenarios and dummy entities removed. With that no migration efforts were made. You may see what happens in savegames when you load the save from version 0.2.0 with 0.3.0.

The version 0.3.0 comes with a single simple scenario, load it up and save the game, we'll use that savegame as an example for migration. As before I've made this scenario through a combination of map editor, savegame converter, and magic of scripting. Here, I've used control.lua to raise fake building events so that teleportation chests got registered by the logic of the mod itself. That script is not directly related to the task at hand, so let's not bring its code here (the file is 50% comment in case you'd want to have a look).

There are also versions 0.3.1, 0.3.2 and 0.3.3 on portal. These versions are results of applying the steps described below to the version 0.3.0. Get them if you get stuck somewhere in this chapter.

So now you have a savegame, let's start changing our mod.

Migrating technologies and recipes

Firstly, instead of two separate recipes for receiver and sender will create a single recipe that takes twice as much resources and gives both of those chests. Yes, I know this is silly, keep moving.

So in data.lua we find the technology and alter its effects:

 --data.lua
local technology={
    type = "technology",
    name = "logistics-teleportation",
    icon = "__base__/graphics/technology/logistics.png",
    effects ={
        {type = "unlock-recipe",
        recipe = "both-teleportation-chests"},
    },
    prerequisites = {"logistics-3", "automation-3"},
    unit = table.deepcopy(data.raw.technology['logistics-3'].unit),
    order = table.deepcopy(data.raw.technology['logistics-3'].order),
}


We also define the new recipe:

local new_recipe={
    type = "recipe",
    name = "both-teleportation-chests",
    enabled = false,
    ingredients ={
        {'advanced-circuit',10}, {'steel-plate',10}, {'electronic-circuit',10},
    },
    results = {
        {type='item', name="teleportation-chest-item",},
        {type='item', name='teleportation-chest-receiver',}
    },
    icon="__First_mod__/icon-1.png", --multiple result recipes need icon
  }

And do not forget to put it into data:

data:extend{technology,receiver_recipe,recipe,item,receiver_item,entity,receiver,power_entity,new_recipe}

For now we'll keep old recipes. Now we rename our mod folder to First_mod_0.3.1 and update info.json accordingly.

When you load the test savegame, take a look at the technologies list, the teleportation technology there shows information from our new definition (It has been researched from the start of the scenario). Yet, if you look into recipes available to you, you'll notice that old separate recipes are there and new one is not present. That is because, for whatever reason, effects of technologies are not updated upon version change. We'll have to do that manually.

So, inside the mod folder (First_mod_0.3.1) we create another folder named migrations and inside it we put a lua file. I've decided to name it 0.3.0to0.3.1.lua, you may come up with another name if you want. After you create a migration file, you'll have to restart the factorio program for it to detect the file. You will not have to do that when you alter the file.

In that lua script we manually check whether the technology is researched, and if it is, we enable the recipe:

for _,f in pairs(game.forces)
    if f.technologies["logistics-teleportation"].researched then
        f.recipes["both-teleportation-chests"].enabled=true
    end
end


Additionally, we might set old recipes to be disabled:

for _,f in pairs(game.forces)
    if f.technologies["logistics-teleportation"].researched then
        f.recipes["both-teleportation-chests"].enabled=true
    end
    f.recipes["teleportation-chest"].enabled=false
    f.recipes['teleportation-chest-receiver'].enabled=false
end


That is, if you still want to keep them in game. Alternatively, you might just erase their definition or registration in the data.lua. These approaches cannot be combined however. If you try to access nonexistent recipe, the game will error.

If you decide to alter technology with different effect, say you want bullet damage researches to be twice as effective, you will have to apply said effects manually as well:

for _,f in pairs(game.forces)
    for n,t in pairs( f.technologies ) do
        if t.researched and n:find("^bullet-damage-") then
            local m= f.get_ammo_damage_modifier('bullet')
            m=m +  t.effects[1].modifier or 0  --at this point technology still has old parameters, so doubling is just another addition (or 0 is simply safety precaution)
            f.set_ammo_damage_modifier('bullet, m)
        end
    end
end


Renaming stuff

Now we'll be migrating the 0.3.1 to 0.3.2 . Suppose we have come up with a better name for something. Let it be that we now want our 'teleportation-chest-item' to be 'teleportation-chest-sender' so that it conforms better with 'teleportation-chest-receiver'. ('teleportation-chest-entity' is going to be a different story, due to our scripts needing references to those entities)

So, firstly, we go through our data.lua script and change the item name everywhere we use it. Note that since 'teleportation-chest-item' was unique string, we can just use simple find and replace command of the editor. If we'd want to rename 'teleportation-chest-receiver'-item then we'd have to keep track of where it is used as an item name and where as an entity name. So naming rules in your mods are up to you.

Then we make a json file in the migrations folder. I've decided to name it '0.3.1to0.3.2.json'. There we'll put the following snippet copied from API documentation:

{
  "item":
  [
    ["teleportation-chest-item", "teleportation-chest-sender"]
  ]
}

So, now we load the same old save (or a newer save with 0.3.1 version) and see the report about replaced items. And... probably no difference whatsoever is noticed for player. It may even appear that the game has detected the renaming and applied locale correctly. Actually it simply pulls localized strings from place_result entity if there's no strings related to item itself. To demonstrate this, the locale was altered in 0.3.2 version from portal.

You could also use the method described here to change name of an entity, or even type of an entity. But not 'teleportation-chest-entity'. As said before, we need those in global, but these json migrations are just a .destroy() followed by creation of new entity. With that you get your references chopped.

Renaming the tracked entities

migrations alone will not help us to migrate from 0.3.2 to 0.3.3 . To rename "teleportation-chest-entity" to "teleportation-chest-sender" we'll have to use the script.on_configuration_changed(). In our mod we we have a global.senders table, where we store references to the discussed entities. A simple .json would invalidate those references. We will need to restore these references after mod update.

Generally (for some other mod), one way of doing such a change would be to use .json migration anyway and then just registering the new stuff.

script.on_configuration_changed(function(changes)
     --see if this mod is upgraded and old version is old
    if changes.mod_changes
        local c=changes.mod_changes['First_mod']
        if c and c.old_version and c.old_version < '0.3.3' then
         --find entities that are updated
            for _,s in pairs(game.surfaces) do
                for _,e in pairs(s.find_entities_filtered{name="teleportation-chest-sender"})
                    if e.valid then
                        add_new_entity_to_global(e) --this function realization is up to you
                    end
                end
            end
        end
    end
end)


If your mod will clean its global from invalid entities and you don't really need any information from them, this will work okay.

The First_mod could get away with this as well. But if you have some serious data associated with entities you'd want to preserve that data. In such case things are going to get uglier. Firstly, you don't simply change prototype's name in data.lua, you make two copies of prototype, one with old name and one with new:


local old_entity=table.deepcopy(entity)
old_entity.order='o-b-s-o-l-e-t-e'
entity.name="teleportation-chest-sender"
data:extend{entity,old_entity}


Do not forget to update items' place results. After you're done with data.lua, you do the replacement manually with the control.lua :

script.on_configuration_changed(function(changes)
     --see if this mod is upgraded and old version is old
    if changes.mod_changes then
        local c=changes.mod_changes['First_mod']
        if c and c.old_version and c.old_version < '0.3.3' then
         --we have references to every entity we need to replace
            for f_name,chests_of_force in pairs(global.senders) do
                local new_chests={}
                for old_uid, pair in pairs(chests_of_force) do
                    local old=pair.chest
                    if old.valid then --trust no one
                        local new=old.surface.create_entity{name="teleportation-chest-sender", position=old.position, force=old.force}
                         --here we copy any important information stored in entity
                             --First_mod has none of that
                         --here we prepare update global (remember we're iterating over it right now)
                        pair.chest=new
                        new_chests[new.unit_number]=pair
                         --and we remove the old entity
                        old.destroy()
                    end
                end
                 --update global
                global.senders[f_name]=new_chests
            end
        end
    end
end)


This variant should also be faster, since it does not involve the search over all surfaces. Not that it'd matter for a one time called function. Obviously, you would also need to check the rest of the script so that id uses new name. (One place easy to forget would be scripts in scenarios if you deal with those.)

Reworking the code

One last thing about the tutorial savegame\scenario. While it is running, press F5. A bunch of numbers will appear on screen. In the lower half of that text you can notice the line that starts with 'mod-First_mod'. This is the average time used by our mod on each tick. The number is milliseconds. The game is designed to run at 60 fps per second. That means each frame shouldn't take more than 16 ms to calculate.

On my machine in that setup, the First_mod takes up to 2.4 ms per tick (You may get different numbers obviously.). That is quite a lot of time considering that in general case there should be other mods and calculations of the base game aren't instant either.

If you remove some of the senders or receivers, you will notice how this update time goes down. With a single pair of chests it goes as low as 0.300 microseconds. For a mod that continuously updates the game state that is not really much. There are mods which go below that however.

So, we need to identity slow parts of our mod. Obviously, those are in the code called in the on_tick handler. And we have also learned that they are related to the total number of our entities.

Factorio doesn't provide many tools for profiling, so in order to gauge what is slow we'll have to simply alter the control.lua, load save and check the update time after modifications. I for example tried commenting lines 52-66. Those are the lines that actually make things teleport. Without them the update time dropped below 0.15.

Now going one step further, let's uncomment line 52, the one where we invoke the closest receiver search. We now see that it accounts for approximately 1 ms out of 4. Normally looping 40 loops over 40 elements would not take that long, but here these loops involve calls to the API functions of factorio.

You may not see them at first, because they're hidden in the lua syntax. But in truth, when we write local p=e.position at line 85, we are invoking a metamethod which calls the C++ function which actually returns you a lua table.

Now this will probably look complicated and is definitely not the way of doing things right, but to just gauge the influence of the API calls let's do the following dirty hack:

 --lines 85-86
local p=e.position
rawset(e,'position',p)

Short explanation would be that e here is just a lua table without any useful fields with all the work being done by metatable. We use rawset to add field to a table, so now when we ask e's position we do not call metamethods. This added field has no influence on the game however, nor it is updated if entity was moved. This is why this should not be done in the release version.

Anyways, the find_nearest was called 40 times per tick and during each call it looped over 40 entities. That gives us 1600 position inquiries. By ruling out those calls to entity metamethod we have lowered time from 2.4 ms to 1.4 ms. We cannot really do the same with validity check on the line 80. 1600 calls to that account for the other about 0.6 ms of the time wasted in search function. This can be checked for example by mangling the line 80 in the following manner:

if false and not e.valid then

Of the remaining 0.8 milliseconds 0.5 goes to the logic calculations inside loop (including calls to rawset which shouldn't even be inside the loop if it was done correctly). And the rest 0.3 is all the other code that is still run on tick (I mean with lines 53-66 commented out).

Apart from bashing the API sluggishness, there's other point of note here: neither API, nor the search function itself are actually that bad, it is us who calls them way more often than needed. Chests are notable for their immobility, so the corresponding receiver will only ever change when some chest is built or removed while we are doing the search on every second.

Let's perform another hack and eliminate the unneeded search :


 --line 100
function cantor_pair(position) --return unique number for each pair (see links in manual for more)
    local x,y = position.x, position.y
    local a= (x >= 0 and (x * 2) or (-x * 2 - 1))
    local b= (y >= 0 and (y * 2) or (-y * 2 - 1))
    return (a + b +1)*(a + b)/2 + a
end
local cache={} --this will cause desyncs in multiplayer
local true_find=find_nearest
find_nearest=function(entities,position,max_distance) --this will redefine the old function
    local id= cantor_pair(position)
    local cached=cache[id]
    if cached and cached.entity.valid then
        return cached.entity, cached.distance
    else
        local e,d = true_find(entities,position,max_distance)
        cache[id]={entity=e, distance=d}
         --line 118
        return e,d
    end
end


What we did here is a replacement of the old slower function by new one, which stores its previous results and, if possible, returns those instead of doing a complete search. That trick did cut additional 0.7 ms from 1.4 we had before. In fact, adding caching brings mod update time to 0.7-0.8 ms without other tricks. (Obviously, it eliminates the unneeded calls metamethods by not calling the function that does that.)

Outside the search function, however, there's not much that can be improved unless we are willing to drop parts of functionality of the mod. We will not do that. We'll have to settle with eliminating calls to .get_inventory(). Let's do that with the same trick as before:

 --line 56
local receiver_inventory=rawget (nearest,'inventory')or nearest.get_inventory(defines.inventory.chest)
rawset(nearest,'inventory',receiver_inventory)
local sender_inventory=rawget(chest,'inventory') or chest.get_inventory(defines.inventory.chest)
rawset(chest,'inventory',sender_inventory)

And voila: another 0.4 ms is gone and mod is now update now commences in about 300 microseconds.

Now there's one final thing to change: we are doing our updates once a second, but when we do - we update all the entities. The game timer however prefers a more even workload spread. If you comment the line 44:

 --line 43
if not (e.tick%60)==0 then return end

you will not see any change in update times. The measurements likely provide the largest update time per 60 ticks or something similar. And there is probably wisdom behind that, people don't like stuttering. So let's try to rig something different here:

 --line 43
local t= e.tick%60
for force_name,receivers in pairs(global.receivers) do
    if global.senders[force_name] then
        local i=0
        for uid,tab in pairs(global.senders[force_name]) do
            i=i+1
            if not ((i%t)==0) then goto skip end

Basically, here we assign some integer number to each iteration of the inner for loop, and on each tick we do only each 60th iteration in the loop and skip others. The t variable provides shift in the selection, so that on ticks 61, 121, etc... we perform updates of entities that are looked at first, sixty-first and so on. To skip the update, we use the goto statement which moves the execution to the label located at the bottom of the for loop:

 --line 75
::skip::

So, this staggering brings our update time to about 0.07 ms. As a side effect, the ingame power statistics became less jittery as well.

There probably are a few of low level tricks that might cut some microseconds if we really get our hands dirty. But nothing groundbreaking anymore. The rest of the code is there for a reason and cannot be skipped.

Now, we have finished exploring the possibilities, but the mess we have made is not a release-grade code. Firstly, the rawset\rawget methods. Those did help us to cut some corners while testing, but you should really think it out before you incorporate such methods into the core of your mod. These methods open up opportunities for a lot of mistakes and oversight.

Secondly, the caching. We surely might make it global.cache instead of local cache and have it transfer through the network correctly. That would make a working code, but not a readable one. And that would cause problems, when someone else (or future you) would try to figure out what happens in the mod.

We have used both the tricks to store some frequently used information. The same information could also be easily stored by using tables that's the reason they exist after all. The control.lua of the First_mod_0.3.4 features the code with the same optimizations but without questionable practices. That version of the mod runs at about 0.03 - 0.04 milliseconds per update. The file has comments inside it, have a look if you are interested. Also, the mod contains "control_profiled.lua" which is a result of our "exploration" from above, have a look at that if you couldn't mangle one yourself.

The chapter is over.

To Boldly Go Where No Man Has Gone Before...

Well, this was quite a story, wasn't it. This tutorial comes to its conclusion. It is wiki, however, maybe someone will come up with new knowledge someday. Go now, make your own stories.