Synchronize with fibers
Instalation of walk and fibers
This time, Let's play with node.js and CoffeeScript.
- [walk] GitHub - solderjs/node-walk: A semi-port of python's os.walk
- [fibers] GitHub - laverdet/node-fibers: Fiber/coroutine support for v8 and node.
Install walk and fibers.
$ mkdir walk_fibers $ cd walk_fibres $ npm install walk fibers $
Asynchronus Directory traverse with walk
walk is a library that proceseses files traversing directory tree recursively. It is also asynchronous so you can gain speed.
- walk.coffee
# node.js modules
Path = require('path')
# npm modules
Walk = require('walk')
class Traverse
constructor: -> @traverse_async('.')
traverse_async: (root_dir) =>
list = []
# *1_1: Start directory traverse.
walk = Walk.walk(root_dir)
# *1_2: Callback on file stat-ed.
walk.on 'file', (root, stat, next) =>
filename = Path.join(root, stat.name)
list.push(filename)
next()
# *1_3: Callback on walk ended.
walk.on 'end', =>
list.sort()
# *1_4: Do something with list *asyncronously* in the walk callback.
console.log(list)
new Traverse
How to work on walk
First, the directory traversing starts with *1_1 .
And register a callback that will be called when the file is stated in *1_2 .
Each time stat is called *1_2 , it adds file path to the array.
It is the *1_3 callback that is called when the directory traversing ends.
After it completes the directory traversing, it sorts the array.
And it get list in *1_4 and put it out in callback function.
Here is the execution:
$ coffee walk.coffee [ 'node_modules/fibers/.npmignore', 'node_modules/fibers/LICENSE', 'node_modules/fibers/README.md', 'node_modules/fibers/bin/.npmignore', 'node_modules/fibers/bin/darwin-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/darwin-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.6/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.14/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.6/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.14/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.6/fibers.node', ... 'walk.coffee', 'walk_fibers.coffee' ] $
Synchronization with fibers
Next, Try also fibers. You can wait for asynchronus directory traversing with it.
- walk_fibers.coffee
# node.js modules
Path = require('path')
# npm modules
Walk = require('walk')
Fiber = require('fibers')
class Traverse
constructor: ->
# *2_1: Register @main_procedure as the main thread and run the main thread.
Fiber(@main_procedure).run()
main_procedure: () =>
list = @traverse('.')
# *2_6: Do something with list *syncronously* in the main procedure.
console.log(list)
traverse: (root_dir) =>
list = []
# *2_2: Get the main thread for later at *2_4.
main_thread = Fiber.current
# Start directory traverse.
walk = Walk.walk(root_dir)
# Callback on file stat-ed.
walk.on 'file', (root, stat, next) =>
filename = Path.join(root, stat.name)
list.push(filename)
next()
# Callback on walk ended.
walk.on 'end', =>
list.sort()
# *2_4: Resume back the the main thread at *2_5.
main_thread.run(list)
# *2_3: Yield the main thread to async walk procedure
list = Fiber.yield()
# *2_5: Will be resumed here.
return list
new Traverse
How to work on fibers
First, register and run main_procedure () as the main thread at *2_1 .
Get the main thread object with *2_2 . It is a preparation to use this object in *2_4 .
Process walks asynchronously with travase().
Control is returned to the main thread during asynchronous processing. Processing of the main thread yields by executing Fiber.yield() with *2_3 . Wait that asynchronous processing ends.
When asynchronous processing is finished, return to the main thread from the walk 'end' function in *2_4 .
After the main thread is synchronized and moves to *2_5 and exits traverse(). It returns to main_procedure() at *2_6 . You will be able to use list to do further processing.
$ coffee walk_fibers.coffee [ 'node_modules/fibers/.npmignore', 'node_modules/fibers/LICENSE', 'node_modules/fibers/README.md', 'node_modules/fibers/bin/.npmignore', 'node_modules/fibers/bin/darwin-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/darwin-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/darwin-x64-v8-3.6/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.14/fibers.node', 'node_modules/fibers/bin/linux-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/linux-x64-v8-3.6/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.11/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.14/fibers.node', 'node_modules/fibers/bin/win32-ia32-v8-3.6/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.11/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.14/fibers.node', 'node_modules/fibers/bin/win32-x64-v8-3.6/fibers.node', ... 'walk.coffee', 'walk_fibers.coffee' ] $
Advantages of fibers
In the latter code, you can further process within the same hierarchy from *2_6 after performing traverse() in main_procedure().
The point is not within the walk 'end' callback, but main procedure continues after asynchronus processing of walk.
