I was making some research for one of upcoming project at my job. I needed to test http server if it can handle 500-1500 concurrent requests.
It's was more hard to make right testing rather then right server. Web server don't have any record in error log and could process requests that I sent with curl while load test is running.
Setup
First I setup Nginx with HttpEchoModule, I used openresty to easily configure and compile with extra modules.
Next I create config to wait 3 seconds and response with some text.
I choose nginx as example to compare with real application.
location / {
add_header Content-Disposition "inline";
add_header Content-Type "text/plain";
echo_sleep 3;
echo "response text";
}
Testing
Then I start to test how many requests it can process at same time:
ab -n 100 -c 100 http://127.0.0.1:7080/
# => ok
ab -n 200 -c 200 http://127.0.0.1:7080/
# => ok
ab -n 500 -c 500 http://127.0.0.1:7080/
# => socket: Too many open files (24)
Probably we hit system limitation on max open files per process.
Read more
I was suprised how nice to get rid of endless nested callbacks in javascript.
I was playing with node-fibers, trying to make it easier to write tests.. One day I came up with idea: what if one function can run asynchronously when we call inside fiber and synchronous when passing a callback function. Here's how I can make it:
var Fiber = require('fibers');
// can be called to convert multiple methods
Fiber.makeSync = function (receiver) {
for (var n = 1; n < arguments.length; n++) {
Fiber.makeSyncFn(receiver, arguments[n]);
}
};
Fiber.makeSyncFn = function(receiver, methodName, errorArgNum) {
var origFn = receiver[methodName];
if (origFn == undefined) {
throw "Object don't have property '" + methodName + "'";
}
receiver[methodName] = function () {
var lastArg = arguments[arguments.length - 1];
// check if it called inside Fiber
if (Fiber.current && typeof lastArg != 'function') {
var fiber = Fiber.current;
var newValue;
var args = Array.prototype.slice.call(arguments);
if (typeof errorArgNum == 'undefined') errorArgNum = 1;
args.push(function(data) {
// retrieve error from arguments (optional)
var error = arguments[errorArgNum];
if (error) {
throw error;
}
// assign result and resume fiber
newValue = errorArgNum == 0 ? arguments[1] : data;
fiber.run();
});
// call original function with fiber-aware callback
origFn.apply(this, args);
// pause and wait till resume
Fiber.yield();
return newValue;
} else {
origFn.apply(this, arguments);
}
};
};
module.exports = Fiber;
Read more
Today I find tool pgloader http://pgloader.io/ to convert database from Mysql to Postgres.
Precompiled package can be installed from downloads page or you can compile by yourself from lisp source on github.
Then run it as:
psql -c "CREATE DATABASE db_name"
pgloader mysql://user:pass@localhost/db_name postgres:///db_name
It shows warning that can't find libsybdb.dylib, press 0 to continue and it working well.
At the end you will see statistics table:
table name read imported errors time
------------------------------ --------- --------- --------- --------------
fetch meta data 18 18 0 18.096s
create, drop 0 14 0 0.285s
------------------------------ --------- --------- --------- --------------
attachments 0 0 0 0.154s
emails 1782 1782 0 2.425s
email_bcc 20 20 0 0.044s
email_cc 0 0 0 0.045s
email_to 1847 1847 0 0.242s
http_notifications 1097 1097 0 0.317s
templates 6 6 0 0.171s
Index Build Completion 0 0 0 0.000s
------------------------------ --------- --------- --------- --------------
Create Indexes 11 11 0 0.237s
Reset Sequences 0 0 0 0.361s
Primary Keys 7 7 0 0.049s
Foreign Keys 4 4 0 0.071s
Comments 0 0 0 0.000s
------------------------------ --------- --------- --------- --------------
Total import time 4752 4752 0 22.260s
Done!
If compare many arguments vs “options” hash, I usually choose options hash. And I often face problems when I pass wrong key name or what not supported.
ActiveSupport already have Hash#assertvalidkeys, I added one more method assert_required_keys:
class Hash
def assert_required_keys(*keys)
keys.flatten.each do |key|
raise ArgumentError.new("Required key: #{key.inspect}") unless has_key?(key)
end
end
end
Here is my solution to validate hash keys:
module ValidateOptions
def validate_options!(params, options)
options[:optional] ||= []
options[:required] ||= []
params = params.deep_symbolize_keys
params.assert_required_keys(options[:required])
params.assert_valid_keys(options[:required] + options[:optional])
end
end
Usage example:
include ValidateOptions
def method_with_options(options = {})
validate_options!(options,
required: [:order_id, :subject, :from, :to, :text_body, :html_body],
optional: [:cc, :bcc, :template_name]
)
end
Here's how to handle exceptions in D.
try {
throw new Exception("error message");
} catch (Exception error) {
writefln("Error catched: %s", error.msg);
} finally {
writefln("in finaly block");
}
catch (Exception error) will catch all exception of type Exception and child (inhereted) types.
Read more
We can use anonymous functions in D.
delegate keyword
Example with argument:
import std.stdio;
void evenNumbers(int[] numbers, void delegate(int) callback) {
foreach (int number; numbers) {
if (number % 2 == 0) callback(number);
}
}
void main() {
auto numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
evenNumbers(numbers, (num) => writeln(num));
// or numbers.evenNumbers((num) => writeln(num));
// or numbers.evenNumbers((num) { writeln(num); });
}
In this case (num) => writeln(num) is an argument, callable function.
Output:
% rdmd delegate.d
2
4
6
8
10
Read more
I'm not big fan of new icon for iTunes 12. Here is how to change it:
iTunes keep it's icon at
/Applications/iTunes.app/Contents/Resources/iTunes.icns
For iTunes 11 you can download here: https://dl.dropboxusercontent.com/u/586048/iTunes.icns
I checked this manual and app liteicon it worked for me.
But it also works if you just replace icon.
# download file
wget https://dl.dropboxusercontent.com/u/586048/iTunes.icns -O /private/tmp/iTunes.icns
cd /Applications/iTunes.app/Contents/Resources
# replace icons
sudo mv iTunes.icns iTunes_red.icns
sudo cp /private/tmp/iTunes.icns .
# drop cache
sudo touch /Applications/iTunes.app
sudo touch /Applications/iTunes.app/Contents/Info.plist
sudo find /private/var/folders/ -name com.apple.dock.iconcache -exec rm {} \;
# restart dock (may take few seconds)
killall Dock
I tested on os x 10.9.5, blue iTunes icon is from macmini having not updated iTunes
In web application have to handle about 4 different timezones:
- application timezone
- user timezone
- server time timezone
- database timezone
Application timezone
Usually same with timezone of your business location, also usually a default timezone.
Rails.configuration.time_zone # => "Jakarta"
# or
MyApp::Application.config.time_zone
User timezone
Timezone of current user, can be saved in user profile, or defected from javascript, eg http://pellepim.bitbucket.org/jstz/
Usually set as: Time.zone
Server timezone
It's a system time of current server, mostly UTC but sometimes can be a time of datacenter location. In development it's developer's machine timezone. You may need it if you want to make a cron task. Cron works in server's timezone.
To calculate it I have handy function what I use with gem whenever:
# Your business timezone
Time.zone = "Asia/Jakarta"
def to_server_time(time_str)
time = Time.zone.parse(time_str)
server_offset = Time.now.getlocal.utc_offset.to_f / 1.hour
time.in_time_zone(server_offset)
end
to_server_time('00:30') # => Sun, 28 Sep 2014 17:30:00 WET +00:00
to_server_time('00:30').strftime("%H:%M") # => "17:30"
Database timezone
Mysql don't store timezone in datetime format, but Postgresql does. Anyway it better if all dates in database have same timezone. Usually it's UTC, or can be custom:
Rails.configuration.active_record.default_timezone
# or
config.active_record.default_timezone
I like node.js because it's very fast. But I also like to make a global objects in my application. I think nothing wrong with it. For example I have global.App object to keep object-wide things.
It was really annoying for me to write require('fs') or require('util') in every file, I also think it's not as fast as access global object.
So I come up with this solution:
var node = {};
var modules = ['child_process', 'fs', 'http', 'https', 'cluster',
'crypto', 'dns', 'domain', 'net', 'url', 'util', 'vm', 'path'
];
var loadedModules = {};
modules.forEach(function(moduleName) {
Object.defineProperty(node, moduleName, {
get: function() {
if (!loadedModules[moduleName]) {
loadedModules[moduleName] = require(moduleName);
}
return loadedModules[moduleName];
}
});
});
module.exports = node;
global.node = node;
It make lazy-loading for all standart libraries as getters of global object node
Later in my code:
node.fs.readFile();
node.util._extend({}, params);
node.url.parse(node.url.resolve(endpoint, filepath));
node.url.format(urlObj);
I'm not sure if Travis CI can install packages, but with drone.io it's possible
Here is how I make it:
- Add project
- Choose C/C++ as language
- Add build command which will install D compiler and run tests.
Command:
# prepare apt repository
sudo wget http://netcologne.dl.sourceforge.net/project/d-apt/files/d-apt.list -O /etc/apt/sources.list.d/d-apt.list
sudo apt-get update && sudo apt-get -y --allow-unauthenticated install --reinstall d-apt-keyring && sudo apt-get update
# install dmd and dub
sudo apt-get install dmd-bin dub
# build and run tests
dub test
Next page