Module:Buffer/doc

This module was originally developed to optimize string concatenation in Module:Asbox but can be used in any module.

The interface for Module:Buffer objects is similar to that of mw.html objects in that you may build complex strings with independent child nodes. In most cases, you may use Buffer objects like a normal string, including using .. operator (though Buffer:_ has the same role, but potentially over 10 times faster than ..).

Buffers can also be appended to mw.html objects via mw.html:node (though not mw.html:wikitext because of type checking). (See also #usage with string/mw.text libraries)

Basic usage

require'Module:Buffer'

require'Module:Buffer'( ... )

require'Module:Buffer'( _G, name, save, ... )

Creates a new Module:Buffer object when it (the module) is called as a function—i.e., there is no 'main'.

Initialize the module with your "local" global variable _G prior to creating any Buffer objects to enable global functions. If passed _G then the next two varargs will pass to Buffer:_G and any extra will pass to Buffer:_. If initialized without _G then all varargs will pass to Buffer:_( ... )


.

You may also use most Buffer object functions directly on the module—i.e require'Module:Buffer':function{...} is equivalent to require'Module:Buffer'():function{...}. The self operator, :, and . are interchangable when used on the Module, but is required for all other interactions with Buffer objects (other than Buffer.last_concat).

Buffer

Buffer( sep, i, j )




See also

Buffer:_str for advanced string conversion.

Get Buffer as type string by performing a function call on the Buffer object (as opposed to a call on the Module). Calling a Buffer is basically shorthand for table.concat( Buffer, ... ) , or, with no args, tostring( Buffer ) .

However, if your Buffer contains raw objects or out-of-sequence values, then the return string would be the result of empty-buffer:_all( Buffer )( ... )


instead.

Buffer.last_concat

When strung without a separator, the result may be retrieved via Buffer.last_concat. Future tostring operations on the Buffer will return Buffer.last_concat until it is modified.

You may purge the cache by setting this key to nil, by appending a valid value and/or removing a value: Buffer:_(0)


:_nil(), as well as by passing nothing to Buffer:_c()


.

Buffer:_

Buffer:_'string'




Buffer:_( value, pos, raw )



Buffer:_( value, raw )




See also

Buffer:stream for a faster, simpler version of this op.

Appends a value to the Buffer. In rough terms, Buffer:_'string1':_'string2' is the same as Buffer = Buffer..'string1'..'string2'. (It may help to imagine :_ as a .. that has stood up and is now casting a shadow.)

If passed an invalid value listed below, this is a no-op:

• boolean
• nil
• empty string
• table without a __tostring metamethod and which table[1] is nil or false.

A table with no __tostring will pass through table.concat before insertion. An error may be thrown if the table would cause table.concat to error. (Use Buffer:_all instead for such tables.) For all other value, the result of tostring( value ) would be inserted so long as it is not an empty string.

When passed pos of type number, the argument is identical to pos for table.insert( table, pos, value ) . In fact, assuming a valid value, Buffer:_( 'string', 1 )


is exactly the same as table.insert( Buffer, 1, 'string' ) .

Unconventionally, a pos of type string is treated as relative to length; that is, Buffer:_( 'string', '-1' )


is equivalent to Buffer:_( 'string', #Buffer - 1 )


(obviating the need to set a local Buffer to use the length operator).

Set raw to true to force value in Buffer without tostring coercion, including invalid values. If given only two (non-self) arguments with the second being a boolean, then the second is read as raw instead.

Buffer:_nil

Buffer:_nil( pos, replacement )




Removes the value buffered at pos. As with Buffer:_, a string pos string is treated as #Buffer + pos.

If replacement is provided, then this will replace the value at the pos index as long as replacement is not a boolean, in which case, this is a no-op.

When replacement is nil, the op is simply table.remove( Buffer, pos ) with string pos relative to length. Note however there is no further type checking on replacement, so, if nil nor boolean, then Buffer will be set to raw.

Pos cannot be omitted if replacement is passed, though a pos that is nil will be treated as #Buffer.

Buffer:_all

Buffer:_all{ value, ... }




Buffer:_all( { ..., value = pos, functionName = args, ... }, nanKeys )




Takes a table value, iterates through all number keys in order, appending each valid value to the end of the Buffer. In contrast to ipairs, this starts at the most negative key (down to -inf), continues through any nil keys, until it reaches the most positive index and includes non-integer number keys. (Note: despite Module:Buffer.__pairs having a more thorough iteration than ipairs, the difference in their runtimes is almost statistically insignificant. Details at #Performance and #Using the iterator outside of buffer.)

A table value that has no metatable will have its contents iterated by this function before moving on to the next value. All other data types are processed by Buffer:_.

By default, this ignores non-number keys unless nanKeys evaluates true. If so, non-number keys are processed after number keys. Keep in mind such keys are iterated in no particular order, though an order may be imposed by wrapping each pair in a table indexed at a number key.

If given a value = pos pair, defined as a number or number string indexed at a non-number key, then they will be passed as the value and pos arguments for Buffer:_. Thus, Buffer:_all({1,2,3,'... done',[3.5]=variable and 4 or {four='1',zero=1}},true)


produces the same result as:

Buffer:_(1):_(2):_(3)
if variable then
	Buffer:_(4)
else
	Buffer:_'four':_('zero',1)
end
Buffer:_'... done'

If a non-number key points to a value that cannot be coerced into a coerced into a number then the pair may be treated as functionName = args, when functionName matches a Buffer object function and args is not boolean. If args is such that value[1] evaluates true, then this will pass the return of unpack( value, 1, table.maxn(value) ) to the named function; otherwise, the value is passed as is. For example:

p(_G,'arg', true):_all({'arg',arg==true and {'==true: ' ,_in={_G, 't', nil, ' awesome'}}}, true):_(t and {t(), t..'r', t..'st'})

produces: 'arg==true: awesome awesomer awesomest'

Buffer:_in

Buffer:_in( ... )




Buffer:_in( _G, name, save, ... )




Creates and returns a new Buffer object. This does not not append the new Buffer to the parent. (See next section)

More precisely, it re-calls the Module:Buffer instance which created the Buffer object with the passed arguments and then adds a reference for new Buffer to allow it to retrieve its parent. Do not pass _G if the Module:Buffer instance was not initialized with global functions enabled.

Note that all Buffer parent references are weak. If a variable is set to the parent and then set to something else, such may trigger immediate garbage collection.

Buffer:_out

Buffer:_out( sep )




Buffer:_out( outs, sep–list, { default–sep, ..., [out] = sep } )




Joins Buffer with sep and appends result to the parent Buffer. Returns the parent. If no parent is found, this is a no-op and returns the same Buffer.

If given more than one (non-self) argument, then the first is read as outs—the number of :_out() operations to perform. Each additional argument in sep-list is applied as sep for that :_out operation. That is, the first sep applies to the current Buffer, the second to its parent Buffer, the third to its grandparent, and so on.

If the last vararg is a table, then table[1] will be applied as the default separator for all nil varargs in sep-list. This table may directly follow outs (i.e. sep-list may be omitted). If it contains other keys, then the sep at key N would apply for the Nth :_out() instead of default-sep. Thus, these two snippets are synonymous: Buffer:_out( 4, nil, nil, nil, ' and ', {', '} )


and Buffer:_out( 4, {', ', [4] = ' and '} )


. Include false in sep-list to indicate no separator applies for that out.

Buffer:_str

Buffer:_str( sep )




Buffer:_str( generations, sep–list, { default–sep, ..., [gen] = sep } )




Joins a Buffer with sep and returns the string. Varargs are handled by the same function as Buffer:_out, which, if provided, this will create a new temporary Buffer and backtrack the number of generations specified, inserting each ancestor in front of its descendants in the temporary Buffer. The sep indexed at generations + 1 will be used as the joiner for the temporary Buffer (unless the first ancestor is reached before the specified number of generations, in which case it is the index following that of the original generation).

Unlike :_out, this does not append the child into the parent. As such, even with the same arguments, it may return a different result than would be obtained from stringing the return of :_out since each parents' sep is not used to join parent and child. Furthermore, the number of generations counted includes the current Buffer, whereas the number of "outs" in Buffer:_out does not.

Buffer:_parent

Buffer:_parent( outs, sep–list, { default–sep, [out] = sep, ...} )




To skip generations without breaking the Buffer chain, see #global functions.

Similar to Buffer:_out except, instead of apending the Buffer to its parent, this calls Buffer:_str on the parent(s) and appends the result. The parent is unaffected by this operation and may still be retrieved via Buffer:_out.

Buffer:getParent

Buffer:getParent( value )




Buffer:getParent( functionName, ... )




Returns parent Buffer, or, if none exists, sets a newly created Buffer as the 'parent' and returns the adopted parent.

Following this Module's waste no () philosophy, arguments are passed to the parent. If passed only one value, this is equivalent to Buffer:getParent():_( value )


. If any varargs are given, functionName must be a string naming a Buffer object function (or #library) to be called on the parent using the varargs.

Also know that here is no 'getChild' method.

Buffer:killParent

Buffer:killParent( ... )




Unsets the parent reference, allowing garbage collection unless there are non-weak references to the parent.

If passed any args, they will be passed to the current parent via Buffer:getParent as a "parting gift". In either case, returns to current Buffer.

Buffer:_c

Buffer:_c( clear, copy )




Nils all keys of the table referenced by clear and unsets its metatable. If no table as clear, this simply purges the cache at Buffer.last_concat.

If given a table to copy, this will duplicate all key-value pairs of copy into clear, passing any value of type table through mw.clone. Any metatable and Buffer parent and raw references are also copied.

If copy is not a table, then it will be inserted as the first index of the cleared table so long as copy is not nil or false.

Stream mode

Buffer:stream

Buffer:stream'string'




Buffer:stream{concat–list}



Buffer:stream(var)




Switches the Buffer to stream mode. In this mode, the Buffer call operation, instead of returning a string, now acts as streamlined version of Buffer:_.

When streaming, you may append a sequence of string (and table) literals with nothing between them (or only ASCII space chars if desired). For example, both A and B will produce identical strings:

local A = p:stream'A string of text may flow''with nothing between each string' 'or perhaps only a space'
	'or even tab and line-break characters''and continue to append individually''for use with a joiner'

local B = p:_'A string of text may flow':_'with nothing between each string' :_ 'or perhaps only a space'
	:_'or even tab and line-break characters':_'and continue to append individually':_'for use with a joiner'

mw.logObject{A==B, A:_str' '}
table#1 {
  true,
  "A string of text may flow with nothing between each string or perhaps only a space or even tab and line-break characters and continue to append individually for use with a joiner",
}

Aside from saving two keystrokes per string, this mode runs about 50 percent faster than :_ (which says a lot considering :_ is much faster than the .. op for long strings). Also, Lua numbers and named variables are too shy to skinny dip in a Buffer stream and must wear parenthesis () as with any function call.

Returning to normal mode
No special action is needed to exit this mode. The normal call to string op is restored upon the use of any regular Buffer function or any operation which coerces the Buffer into a string.

HTML extension

Buffer:_inHTML

Buffer:_inHTML( tagName, args )




Buffer:_inHTML{ args–list }




Creates and returns a modified mw.html object. Accepts the same parameters as mw.html.create.

Modifications are summarized below:

• The .. may be used on Buffer-mw.html objects directly (no tostring needed).
• If initialized, will store tags and wikitext in an Element-Buffer, with which you may use Module:Buffer object functions to append (and remove, etc.) values.
• Element-Buffer objects may use Element-Buffer:_add, which greatly reduces the code size needed to build an equivalent mw.html object.

Unlike mw.html.create, if args has keys other than args.parent and args.selfClosing, it will pass through Element-BufferBuffer:_add for further processing. Moreover, if passed a table where mw.html.create expects tagName, this treats it as args instead.

Most mw.html functions are unchanged, except :tag, :done, and :allDone are embedded in a wrapper function that checks whether they return a normal mw.html object. If so, converts it to a Buffer-HTML object and sets a parent reference.

Note that other functions in section #HTML extension are only available after Buffer:_inHTML is used for the first time.

Buffer-HTML

Buffer-HTML'string'




Buffer-HTML{ wikitext–string, { tag = { tagName, arg = value, wikitext, htmlFunction = args, cssName = cssValue } }, ... }




Buffer-HTML objects may be used like any mw.html object. (In fact, if the only change were to substitute mw.html.create with require'Module:Buffer':_inHTML in an existing Module, its output should remain the same.)

Call the object as a function to return its Element–Buffer, which is the table found at mw.html–object.nodes converted into a Module:Buffer object.

Strings are passed to the Element-Buffer via Buffer:_ which basically has the same effect as though :wikitext were between Buffer-HTML and 'string', the only difference being the object returned. Tables are passed to Element–Buffer:_add.

Most Buffer object functions are either unavailable for use directly on the Buffer-HTML object. Those listed below have been modified so that

Element-Buffer

Element–Buffer( sep, i , j )




Element-Buffers have the same metatable as normal Buffer objects, so calling it will string it in the same manner.

The string returned is analogous to the value returned by the "innerHTML" DOM property in JavaScript. In other words, when strung, it is the contents of the Buffer-HTML object without the "outerHTML" or tag.

You may use most Buffer object function normally, however those which have a Buffer-HTML version (such as Buffer-HTML:_out) will instead behave as though used on the outer HTML object. Also, pre-Element:_inHTML has been modified as described in that section.

Additionally, you may chain any mw.html object function directly on an Element-Buffer. With the exception of Element–Buffer:tag and Element–Buffer:done, the mw.html function has been placed in a wrapper function that merely redirects the self-action to the outside Buffer-HTML.

Element-Buffer:done

Element–Buffer( dones )




When called without arguments, this behaves just like mw.html:done as called on the outer HTML object.

However, it has been modified to accept dones, the number of :done() operations to perform. Thus, Element–Buffer:done(4) is equivalent to Buffer–HTML:done():done():done():done().

Pass zero (0) as dones to return to the Element-Buffer's direct HTML container. (Using an mw.html function to no-op is another way to return to the Buffer-HTML object, e.g. Element–Buffer:node()


, though that example does not work for selfClosing tags.)

ipairs with HTML-Buffer

See also #Using 'all' pairs outside of buffer for more details about Module:Buffer's custom iterator.

BufferHTML = p:_inHTML'td'{1,2,nil, '', true, 3,4,tag='br'}:done(0)
mw.log(BufferHTML)
for k, v in ipairs(BufferHTML) do mw.log(k,v) end
for k, v in ipairs(BufferHTML) do if v=='3' then BufferHTML():_nil(k) end end
mw.log(BufferHTML)
<td>1234<br /></td>
1	1
2	2
3	3
4	4
5	<br />
<td>124<br /></td>

Modified .. operator

Buffer .. value
Buffer-HTML .. value
This is akin to '''new-buffer'':_all{ Buffer, value}


or tostring( Buffer ) .. value. HTML objects created by a Buffer may also be concatenated in this manner.

Buffer-HTML .. value

value .. Element-Buffer

Tips and style recommendations

• If joining Buffer with a string immediately after :_'text', place a space between 'string' and the separator and use double/single quote marks to . (i.e. :_'text' " " instead of :_'text'' ' or :_'text'(' '))
• Saving Module:Buffer locally, e.g. local Buffer = require'Module:Buffer' , though fine, is often unnecessary since all Buffer objects can create new buffers via

Buffer:_in.

For Buffer:_

• Treat :_ as though it were a .. op. Wrapping strings with unnecessary () is akin to ( 'string1' ) .. ( 'string2' ) .. ( 'string3' ).
• Most uses of raw can be avoided through careful planning with the pos argument. That said, the performance decrease from raw is unlikely to be significant for modules transcluded on less 100,000 pages. In short, reduction in server load from avoiding raw may not be worth it if such makes the code harder to maintain.
• To insert an empty string as a placeholder for a separator without setting raw, pass a table containing only a empty string, like so:

Buffer:_{''}


.

For Buffer:_all

• Appending values in multiple locations is one of the primary reasons why the nanKeys argument exists. While passing a boolean directly will cause an error, you can do something like...

this: Buffer:_all({condition and {_nil={'0', 'replacement'},Front=1,getParent='from child'}}}, true)

versus: Buffer:_nil('0', condition and 'replacement' or false):_(condition and 'Front', 1):getParent(condition and 'from child'):_B(child).

For Buffer:_c

• If the table reference passed as clear was appended raw in multiple positions, this is akin to performing

Buffer:_nil at all positions simultaneously. (May be easier than trying to come up with a string.gsub pattern)

• Inserting a named empty table is raw as a placeholder to be populated later via this function may be easier than calculating pos argument of

Buffer:_.

For Buffer:_inHTML

• When appending simple HTML structures, something like

Buffer:_( mw.html.create'br' )


is roughly 6 times more efficient than Buffer:_inHTML'br':_out(), at least in terms of server CPU usage. (Though Buffer:_'<br />' is 25 and 4 times more efficient, respectively. Also note that Buffer:_inHTML is slower on the first run due to initialization. After the first run, the efficiency ratio of using mw.html.create directly over Buffer:_inHTML drops to 2.)