Results for hash-switch

Load CSS and JS faster. Minifies, bundles, and optimizes CSS/JS assets ahead of time (e.g., on deploy), not at runtime. Loads popular third-party JS (like jQuery, YUI, and Dojo) from localhost in development, and auto-switches to Google's CDN in production. Lets you switch on LABjs mode to load more scripts in parallel. Can rewrite stylesheets to use CDN hosts (not just your web server) and cache-busting hashes for updated images.

# Trope **[Documentation][docs] - [Gem][gems] - [Source][source]** Prototyping language that transcompiles into pure Ruby code. 1. Build your concept in Trope. 2. Write specs. 3. Transcompile into Ruby. 4. Destroy Trope files. 5. Red, green, refactor. ## Install > NOTE: Trope is not released yet, the gem is just a placeholder. ### Bundler: `gem 'trope'` ### RubyGems: `gem install trope` ## Example Create `library.trope`: ```ruby object Book attr name <String> -!wd 'Unnamed book' attr isbn <Integer> -w attr library <Library> -w do before write { @library.books.delete(self) unless @library.nil? } after write { @library.books.push(self) unless @library.books.include?(self) } end end object Library attr books <Array> -d Array.new meth add_book do |attributes_or_book <Hash, Book>| book = attributes_or_book.is_a?(Book) ? attributes_or_book : Book.new(attributes_or_book) book.library = self @books << book end end ``` Now generate the Ruby code: ```sh $ trope compile libary.trope ``` Those 15 lines will be transcompiled into the following pure Ruby code in `library.rb`: ```ruby class Book class Error < RuntimeError; end class InvalidAttributesError < Error def to_s 'attributes must be a Hash or respond to #to_h' end end class MissingAttributeError < Error def initialize(attr_name, attr_class) @name, @class = attr_name.to_s, attr_class.to_s end def to_s "attribute '#@name' does not exist for #@class" end end class MissingNameError < Error def to_s 'name cannot be nil' end end class InvalidNameError < Error def to_s 'name must be an instance of String or respond to :to_s' end end class InvalidIsbnError < Error def to_s 'isbn must be an instance of Integer or respond to :to_i' end end class MissingLibraryError < Error def to_s 'library cannot be nil' end end class InvalidLibraryError < Error def to_s 'library must be an instance of Library' end end attr_reader *(@@_attributes = [:name, :isbn, :library]) def initialize(attributes={}) raise InvalidAttributesError unless attributes.is_a?(Hash) || attributes.respond_to?(:to_h) attributes = attributes.to_h unless attributes.is_a?(Hash) raise MissingNameError if attributes.has_key?(:name) && attributes[:name].nil? attributes[:name] = 'Unnamed book' unless attributes.has_key?(:name) attributes.each do |name, value| raise MissingAttributeError.new(name, self.class) unless @@_attributes.include?(name.to_sym) setter_method = "#{name}=" setter_method = "_#{setter_method}" unless self.class.method_defined?(setter_method) send(setter_method, value) end end def name=(value) raise MissingNameError if value.nil? raise InvalidNameError unless value.is_a?(String) || value.respond_to?(:to_s) value = value.to_i unless value.is_a?(Integer) @name = value end def isbn=(value) raise InvalidIsbnError unless value.is_a?(Integer) || value.respond_to?(:to_i) value = value.to_i unless value.is_a?(Integer) @isbn = value end def library=(value) raise InvalidLibraryError unless value.is_a?(Library) || value.nil? @library.books.delete(self) unless @library.nil? @library = value @library.books.push(self) unless @library.books.include?(self) @library end end class Library class Error < RuntimeError; end class InvalidAttributesError < Error def to_s 'attributes must be an instance of Hash or respond to #to_h' end end class MissingAttributeError < Error def initialize(attr_name, attr_class) @name, @class = attr_name.to_s, attr_class.to_s end def to_s "attribute '#@name' does not exist for #@class" end end class InvalidBooksError < Error def to_s 'books must be an instance of Array or respond to #to_a' end end attr_reader *(@@_attributes = [:books]) def initialize(attributes={}) raise InvalidAttributesError unless attributes.is_a?(Hash) || attributes.respond_to?(:to_h) attributes = attributes.to_h unless attributes.is_a?(Hash) attributes[:books] = Array.new unless attributes.has_key?(:books) attributes.each do |name, value| raise MissingAttributeError.new(name, self.class) unless @@_attributes.include?(name.to_sym) setter_method = "#{name}=" setter_method = "_#{setter_method}" unless self.class.method_defined?(setter_method) send(setter_method, value) end end def add_book(attributes_or_book={}) raise InvalidAttributesError unless attributes_or_book.is_a?(Hash) || attributes_or_book.respond_to?(:to_h) || attributes_or_book.is_a?(Book) attributes_or_book = attributes_or_book.to_h unless attributes_or_book.is_a?(Hash) || attributes_or_book.is_a?(Book) book = attributes_or_book.is_a?(Book) ? attributes_or_book : Book.new(attributes_or_book) book.library = self @books << book end protected def _books=(value) raise InvalidBooksError unless value.is_a?(Array) || value.respond_to?(:to_a) value = value.to_a unless value.is_a?(Array) @books = value end end ``` Using the transcompiled Ruby code will produce the expected results: ```ruby p library = Library.new # => #<Library:0x007fc55c0ce418 @books=[]> p library.add_book name: 'Book 1', isbn: 1 # => [#<Book:0x007fc55c0cde78 @name=0, @isbn=1, @library=#<Library:0x007fc55c0ce418 @books=[...]>>] p library # => #<Library:0x007fc55c0ce418 @books=[#<Book:0x007fc55c0cde78 @name=0, @isbn=1, @library=#<Library:0x007fc55c0ce418 ...>>]> p library.books.first # => #<Book:0x007fc55c0cde78 @name=0, @isbn=1, @library=#<Library:0x007fc55c0ce418 @books=[#<Book:0x007fc55c0cde78 ...>]>> p library.books.first.isbn = nil # => nil p library.books.first.name = nil # => Book::MissingNameError: name cannot be nil p library.books.first.library = nil # => Book::MissingLibraryError: library cannot be nil p library.books.first.isbn = ['array'] # => Book::InvalidIsbnError: isbn must be an instance of Integer or respond to :to_i p library = Library.new(books: 123) # => Library::InvalidBooksError: books must be an instance of Array or respond to #to_a ``` ### Breakdown ```ruby object Book attr name <String> -!wd 'Unnamed book' end ``` This says that I have an object `Book` that has an attribute `name` (`attr name`) that must either be an instance/subclass of `String` or be able to convert to an instance of `String` using `#to_s` (`<String>`). It is a required attribute that can never be set to nil (`!`), has a writer method (`w`), and defaults to 'Unnamed book'. The minus sign (`-`) indicates a 'switch' or 'option', must like most *nix command line programs. The example could also have been written like so: ```ruby object Book attr name <String> -! -w -d 'Unnamed book' end ``` The above examples will transcompile into the following: ```ruby class Book class Error < RuntimeError; end class InvalidAttributesError < Error def to_s 'attributes must be a Hash or respond to #to_h' end end class MissingAttributeError < Error def initialize(attr_name, attr_class) @name, @class = attr_name.to_s, attr_class.to_s end def to_s "attribute '#@name' does not exist for #@class" end end class MissingNameError < Error def to_s 'name cannot be nil' end end class InvalidNameError < Error def to_s 'name must be an instance of String or respond to :to_s' end end attr_reader *(@@_attributes = [:name]) @@_required_attributes = [:name] def initialize(attributes={}) raise InvalidAttributesError unless attributes.is_a?(Hash) || attributes.respond_to?(:to_h) attributes = attributes.to_h unless attributes.is_a?(Hash) raise MissingNameError if attributes.has_key?(:name) && attributes[:name].nil? attributes[:name] = 'Unnamed book' unless attributes.has_key?(:name) attributes.each do |name, value| raise MissingAttributeError.new(name, self.class) unless @@_attributes.include?(name.to_sym) setter_method = "#{name}=" setter_method = "_#{setter_method}" unless self.class.method_defined?(setter_method) send(setter_method, value) end end def name=(value) raise MissingNameError if value.nil? raise InvalidNameError unless value.is_a?(String) || value.respond_to?(:to_s) value = value.to_i unless value.is_a?(Integer) @name = value end end ``` ## Contributing * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet * Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it * Fork the project * Start a feature/bugfix branch * Commit and push until you are happy with your contribution * Make sure to add tests for it. This is important so I don't break it in a future version unintentionally. * Please try not to mess with the Rakefile, VERSION, or Gemfile. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it. ## Copyright Copyright © 2012 Ryan Scott Lewis <ryan@rynet.us>. The MIT License (MIT) - See LICENSE for further details. [docs]: http://rubydoc.info/gems/trope/frames [gems]: https://rubygems.org/gems/trope [source]: https://github.com/RyanScottLewis/trope

Ame Ame provides a simple command-line interface API for Ruby¹. It can be used to provide both simple interfaces like that of ‹rm›² and complex ones like that of ‹git›³. It uses Ruby’s own classes, methods, and argument lists to provide an interface that is both simple to use from the command-line side and from the Ruby side. The provided command-line interface is flexible and follows commond standards for command-line processing. ¹ See http://ruby-lang.org/ ² See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html ³ See http://git-scm.com/docs/ § Usage Let’s begin by looking at two examples, one where we mimic the POSIX¹ command-line interface to the ‹rm› command. Looking at the entry² in the standard, ‹rm› takes the following options: = -f. = Do not prompt for confirmation. = -i. = Prompt for confirmation. = -R. = Remove file hierarchies. = -r. = Equivalent to /-r/. It also takes the following arguments: = FILE. = A pathname or directory entry to be removed. And actually allows one or more of these /FILE/ arguments to be given. We also note that the ‹rm› command is described as a command to “remove directory entries”. ¹ See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html ² See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html Let’s turn this specification into one using Ame’s API. We begin by adding a flag for each of the options listed above: class Rm &lt; Ame::Root flag 'f', '', false, 'Do not prompt for confirmation' flag 'i', '', nil, 'Prompt for confirmation' do |options| options['f'] = false end flag 'R', '', false, 'Remove file hierarchies' flag 'r', '', nil, 'Equivalent to -R' do |options| options['r'] = true end A flag¹ is a boolean option that doesn’t take an argument. Each flag gets a short and long name, where an empty name means that there’s no corresponding short or long name for the flag, a default value (true, false, or nil), and a description of what the flag does. Each flag can also optionally take a block that can do further processing. In this case we use this block to modify the Hash that maps option names to their values passed to the block to set other flags’ values than the ones that the block is associated with. As these flags (‘i’ and ‘r’) aren’t themselves of interest, their default values have been set to nil, which means that they won’t be included in the Hash that maps option names to their values when passed to the method. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#flag-class-method There are quite a few other kinds of options besides flags that can be defined using Ame, but flags are all that are required for this example. We’ll get to the other kinds in later examples. Next we add a “splus” argument. splus 'FILE', String, 'File to remove' A splus¹ argument is like a Ruby “splat”, that is, an Array argument at the end of the argument list to a method preceded by a star, except that a splus requires at least one argument. A splus argument gets a name for the argument (‹FILE›), the type of argument it represents (String), and a description. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#splus-class-method Then we add a description of the command (method) itself: description 'Remove directory entries' Descriptions¹ will be used in help output to assist the user in using the command. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#description-class-method Finally, we add the Ruby method that’ll implement the command (all preceding code included here for completeness): class Rm &lt; Ame::Root version '1.0.0' flag 'f', '', false, 'Do not prompt for confirmation' flag 'i', '', nil, 'Prompt for confirmation' do |options| options['f'] = false end flag 'R', '', false, 'Remove file hierarchies' flag 'r', '', nil, 'Equivalent to -R' do |options| options['r'] = true end splus 'FILE', String, 'File to remove' description 'Remove directory entries' def rm(files, options = {}) require 'fileutils' FileUtils.send options['R'] ? :rm_r : :rm, [first] + rest, :force =&gt; options['f'] end end Actually, another bit of code was also added, namely version '1.0.0' This sets the version¹ String of the command. This information is used when the command is invoked with the “‹--version›” flag. This flag is automatically added, so you don’t need to add it yourself. Another flag, “‹--help›”, is also added automatically. When given, this flag’ll make Ame output usage information of the command. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#version-class-method To actually run the command, all you need to do is invoke Rm.process This’ll invoke the command using the command-line arguments stored in ‹ARGV›, but you can also specify other ones if you want to: Rm.process 'rm', %w[-r /tmp/*] The first argument to #process¹ is the name of the method to invoke, which defaults to ‹File.basename($0)›, and the second argument is an Array of Strings that should be processed as command-line arguments passed to the command. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#process-class-method If you’d store the complete ‹Rm› class defined above in a file called ‹rm› and add ‹#! /usr/bin/ruby -w› at the beginning and ‹Rm.process› at the end, you’d have a fully functional ‹rm› command (after making it executable). Let’s see it in action: % rm --help Usage: rm [OPTIONS]... FILE... Remove directory entries Arguments: FILE... File to remove Options: -R Remove file hierarchies -f Do not prompt for confirmation --help Display help for this method -i Prompt for confirmation -r Equivalent to -R --version Display version information % rm --version rm 1.0.0 Some commands are more complex than ‹rm›. For example, ‹git›¹ has a rather complex command-line interface. We won’t mimic it all here, but let’s introduce the rest of the Ame API using a fake ‹git› clone as an example. ¹ See http://git-scm.com/docs/ ‹Git› uses sub-commands to achieve most things. Implementing sub-commands with Ame is done using a “dispatch”. We’ll discuss dispatches in more detail later, but suffice it to say that a dispatch delegates processing to a child class that’ll handle the sub-command in question. We begin by defining our main ‹git› command using a class called ‹Git› under the ‹Git::CLI› namespace: module Git end class Git::CLI &lt; Ame::Root version '1.0.0' class Git &lt; Ame::Class description 'The stupid content tracker' def initialize; end We’re setting things up to use the ‹Git› class as a dispatch in the ‹Git::CLI› class. The description on the ‹initialize› method will be used as a description of the ‹git› dispatch command itself. Next, let’s add the ‹format-patch›¹ sub-command: description 'Prepare patches for e-mail submission' flag ?n, 'numbered', false, 'Name output in [PATCH n/m] format' flag ?N, 'no-numbered', nil, 'Name output in [PATCH] format' do |options| options['numbered'] = false end toggle ?s, 'signoff', false, 'Add Signed-off-by: line to the commit message' switch '', 'thread', 'STYLE', nil, Ame::Types::Enumeration[:shallow, :deep], 'Controls addition of In-Reply-To and References headers' flag '', 'no-thread', nil, 'Disables addition of In-Reply-To and Reference headers' do |options, _| options.delete 'thread' end option '', 'start-number', 'N', 1, 'Start numbering the patches at N instead of 1' multioption '', 'to', 'ADDRESS', String, 'Add a To: header to the email headers' optional 'SINCE', 'N/A', 'Generate patches for commits after SINCE' def format_patch(since = '', options = {}) p since, options end ¹ See http://git-scm.com/docs/git-format-patch/ We’re using quite a few new Ame commands here. Let’s look at each in turn: toggle ?s, 'signoff', false, 'Add Signed-off-by: line to the commit message' A “toggle”¹ is a flag that also has an inverse. Beyond the flags ‘s’ and “signoff”, the toggle also defines “no-signoff”, which will set “signoff” to false. This is useful if you want to support configuration files that set “signoff”’s default to true, but still allow it to be overridden on the command line. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#toggle-class-method When using the short form of a toggle (and flag and switch), multiple ones may be juxtaposed after the initial one. For example, “‹-sn›” is equivalent to “‹-s -n›” to “git format-patch›”. switch '', 'thread', 'STYLE', nil, Ame::Types::Enumeration[:shallow, :deep], 'Controls addition of In-Reply-To and References headers' A “switch”¹ is an option that takes an optional argument. This allows you to have separate defaults for when the switch isn’t present on the command line and for when it’s given without an argument. The third argument to a switch is the name of the argument. We’re also introducing a new concept here in ‹Ame::Types::Enumeration›. An enumeration² allows you to limit the allowed input to a set of Symbols. An enumeration also has a default value in the first item to its constructor (which is aliased as ‹.[]›). In this case, the “thread” switch defaults to nil, but, when given, will default to ‹:shallow› if no argument is given. If an argument is given it must be either “shallow” or “deep”. A switch isn’t required to take an enumeration as its argument default and can take any kind of default value for its argument that Ame knows how to handle. We’ll look at this in more detail later, but know that the type of the default value will be used to inform Ame how to parse a command-line argument into a Ruby value. An argument to a switch must be given, in this case, as “‹--thread=deep›” on the command line. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#switch-class-method ² See http://disu.se/software/ame-1.0/api/user/Ame/Types/Enumeration/ option '', 'start-number', 'N', 1, 'Start numbering the patches at N instead of 1' An “option”¹ is an option that takes an argument. The argument must always be present and may be given, in this case, as “‹--start-number=2›” or “‹--start-number 2›” on the command line. For a short-form option, anything that follows the option is seen as an argument, so assuming that “start-number” also had a short name of ‘S’, “‹-S2›” would be equivalent to “‹-S 2›”, which would be equivalent to “‹--start-number 2›”. Note that “‹-snS2›” would still work as expected. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#option-class-method multioption '', 'to', 'ADDRESS', String, 'Add a To: header to the email headers' A “multioption”¹ is an option that takes an argument and may be repeated any number of times. Each argument will be added to an Array stored in the Hash that maps option names to their values. Instead of taking a default argument, it takes a type for the argument (String, in this case). Again, types are used to inform Ame how to parse command-line arguments into Ruby values. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#multioption-class-method optional 'SINCE', 'N/A', 'Generate patches for commits after SINCE' An “optional”¹ argument is an argument that isn’t required. If it’s not present on the command line it’ll get its default value (the String ‹'N/A'›, in this case). ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#optional-class-method We’ve now covered all kinds of options and one new kind of argument. There are three more types of argument (one that we’ve already seen and two new) that we’ll look into now: “argument”, “splat”, and “splus”. description 'Annotate file lines with commit information' argument 'FILE', String, 'File to annotate' def annotate(file) p file end An “argument”¹ is an argument that’s required. If it’s not present on the command line, an error will be raised (and by default reported to the terminal). As it’s required, it doesn’t take a default, but rather a type. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#argument-class-method description 'Add file contents to the index' splat 'PATHSPEC', String, 'Files to add content from' def add(paths) p paths end A “splat”¹ is an argument that’s not required, but may be given any number of times. The type of a splat is the type of one argument and the type of a splat as a whole is an Array of values of that type. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#splat-class-method description 'Display gitattributes information' splus 'PATHNAME', String, 'Files to list attributes of' def check_attr(paths) p paths end A “splus”¹ is an argument that’s required, but may also be given any number of times. The type of a splus is the type of one argument and the type of a splus as a whole is an Array of values of that type. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#splus-class-method Now that we’ve seen all kinds of options and arguments, let’s look on an additional tool at our disposal, the dispatch¹. class Remote &lt; Ame::Class description 'Manage set of remote repositories' def initialize; end description 'Shows a list of existing remotes' flag 'v', 'verbose', false, 'Show remote URL after name' def list(options = {}) p options end description 'Adds a remote named NAME for the repository at URL' argument 'name', String, 'Name of the remote to add' argument 'url', String, 'URL to the repository of the remote to add' def add(name, url) p name, url end end ¹ See http://disu.se/software/ame-1.0/api/user/Ame/Class#dispatch-class-method Here we’re defining a child class to Git::CLI::Git called “Remote” that doesn’t introduce anything new. Then we set up the dispatch: dispatch Remote, :default =&gt; 'list' This adds a method called “remote” to Git::CLI::Git that will dispatch processing of the command line to an instance of the Remote class when “‹git remote›” is seen on the command line. The “remote” method expects an argument that’ll be used to decide what sub-command to execute. Here we’ve specified that in the absence of such an argument, the “list” method should be invoked. We add the same kind of dispatch to Git under Git::CLI: dispatch Git and then we’re done. Here’s all the previous code in its entirety: module Git end class Git::CLI &lt; Ame::Root version '1.0.0' class Git &lt; Ame::Class description 'The stupid content tracker' def initialize; end description 'Prepare patches for e-mail submission' flag ?n, 'numbered', false, 'Name output in [PATCH n/m] format' flag ?N, 'no-numbered', nil, 'Name output in [PATCH] format' do |options| options['numbered'] = false end toggle ?s, 'signoff', false, 'Add Signed-off-by: line to the commit message' switch '', 'thread', 'STYLE', nil, Ame::Types::Enumeration[:shallow, :deep], 'Controls addition of In-Reply-To and References headers' flag '', 'no-thread', nil, 'Disables addition of In-Reply-To and Reference headers' do |options, _| options.delete 'thread' end option '', 'start-number', 'N', 1, 'Start numbering the patches at N instead of 1' multioption '', 'to', 'ADDRESS', String, 'Add a To: header to the email headers' optional 'SINCE', 'N/A', 'Generate patches for commits after SINCE' def format_patch(since = '', options = {}) p since, options end description 'Annotate file lines with commit information' argument 'FILE', String, 'File to annotate' def annotate(file) p file end description 'Add file contents to the index' splat 'PATHSPEC', String, 'Files to add content from' def add(paths) p paths end description 'Display gitattributes information' splus 'PATHNAME', String, 'Files to list attributes of' def check_attr(paths) p paths end class Remote &lt; Ame::Class description 'Manage set of remote repositories' def initialize; end description 'Shows a list of existing remotes' flag 'v', 'verbose', false, 'Show remote URL after name' def list(options = {}) p options end description 'Adds a remote named NAME for the repository at URL' argument 'name', String, 'Name of the remote to add' argument 'url', String, 'URL to the repository of the remote to add' def add(name, url) p name, url end end dispatch Remote, :default =&gt; 'list' end dispatch Git end If we put this code in a file called “git” and add ‹#! /usr/bin/ruby -w› at the beginning and ‹Git::CLI.process› at the end, you’ll have a very incomplete git command-line interface on your hands. Let’s look at what some of its ‹--help› output looks like: % git --help Usage: git [OPTIONS]... METHOD [ARGUMENTS]... The stupid content tracker Arguments: METHOD Method to run [ARGUMENTS]... Arguments to pass to METHOD Options: --help Display help for this method --version Display version information Methods: add Add file contents to the index annotate Annotate file lines with commit information check-attr Display gitattributes information format-patch Prepare patches for e-mail submission remote Manage set of remote repositories % git format-patch --help Usage: git format-patch [OPTIONS]... [SINCE] Prepare patches for e-mail submission Arguments: [SINCE=N/A] Generate patches for commits after SINCE Options: -N, --no-numbered Name output in [PATCH] format --help Display help for this method -n, --numbered Name output in [PATCH n/m] format --no-thread Disables addition of In-Reply-To and Reference headers -s, --signoff Add Signed-off-by: line to the commit message --start-number=N Start numbering the patches at N instead of 1 --thread[=STYLE] Controls addition of In-Reply-To and References headers --to=ADDRESS* Add a To: header to the email headers % git remote --help Usage: git remote [OPTIONS]... [METHOD] [ARGUMENTS]... Manage set of remote repositories Arguments: [METHOD=list] Method to run [ARGUMENTS]... Arguments to pass to METHOD Options: --help Display help for this method Methods: add Adds a remote named NAME for the repository at URL list Shows a list of existing remotes § API The previous section gave an introduction to the whole user API in an informal and introductory way. For an indepth reference to the user API, see the {user API documentation}¹. ¹ See http://disu.se/software/ame-1.0/api/user/Ame/ If you want to extend the API or use it in some way other than as a command-line-interface writer, see the {developer API documentation}¹. ¹ See http://disu.se/software/ame-1.0/api/developer/Ame/ § Financing Currently, most of my time is spent at my day job and in my rather busy private life. Please motivate me to spend time on this piece of software by donating some of your money to this project. Yeah, I realize that requesting money to develop software is a bit, well, capitalistic of me. But please realize that I live in a capitalistic society and I need money to have other people give me the things that I need to continue living under the rules of said society. So, if you feel that this piece of software has helped you out enough to warrant a reward, please PayPal a donation to now@disu.se¹. Thanks! Your support won’t go unnoticed! ¹ Send a donation: https://www.paypal.com/cgi-bin/webscr?cmd=_donations&amp;business=now@disu.se&amp;item_name=Ame § Reporting Bugs Please report any bugs that you encounter to the {issue tracker}¹. ¹ See https://github.com/now/ame/issues § Authors Nikolai Weibull wrote the code, the tests, the documentation, and this README. § Licensing Ame is free software: you may redistribute it and/or modify it under the terms of the {GNU Lesser General Public License, version 3}¹ or later², as published by the {Free Software Foundation}³. ¹ See http://disu.se/licenses/lgpl-3.0/ ² See http://gnu.org/licenses/ ³ See http://fsf.org/

README ====== This is a simple API to evaluate information retrieval results. It allows you to load ranked and unranked query results and calculate various evaluation metrics (precision, recall, MAP, kappa) against a previously loaded gold standard. Start this program from the command line with: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> The options are outlined when you pass no arguments and just call retreval You will find further information in the RDOC documentation and the HOWTO section below. If you want to see an example, use this command: retreval -l example/gold_standard.yml -q example/query_results.yml -f yaml -v INSTALLATION ============ If you have RubyGems, just run gem install retreval You can manually download the sources and build the Gem from there by `cd`ing to the folder where this README is saved and calling gem build retreval.gemspec This will create a gem file called which you just have to install with `gem install <file>` and you're done. HOWTO ===== This API supports the following evaluation tasks: - Loading a Gold Standard that takes a set of documents, queries and corresponding judgements of relevancy (i.e. "Is this document relevant for this query?") - Calculation of the _kappa measure_ for the given gold standard - Loading ranked or unranked query results for a certain query - Calculation of _precision_ and _recall_ for each result - Calculation of the _F-measure_ for weighing precision and recall - Calculation of _mean average precision_ for multiple query results - Calculation of the _11-point precision_ and _average precision_ for ranked query results - Printing of summary tables and results Typically, you will want to use this Gem either standalone or within another application's context. Standalone Usage ================ Call parameters --------------- After installing the Gem (see INSTALLATION), you can always call `retreval` from the commandline. The typical call is: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> Where you have to define the following options: - `gold-standard-file` is a file in a specified format that includes all the judgements - `query-results` is a file in a specified format that includes all the query results in a single file - `format` is the format that the files will use (either "yaml" or "plain") - `output-prefix` is the prefix of output files that will be created Formats ------- Right now, we focus on the formats you can use to load data into the API. Currently, we support YAML files that must adhere to a special syntax. So, in order to load a gold standard, we need a file in the following format: * "query" denotes the query * "documents" these are the documents judged for this query * "id" the ID of the document (e.g. its filename, etc.) * "judgements" an array of judgements, each one with: * "relevant" a boolean value of the judgment (relevant or not) * "user" an optional identifier of the user Example file, with one query, two documents, and one judgement: - query: 12th air force germany 1957 documents: - id: g5701s.ict21311 judgements: [] - id: g5701s.ict21313 judgements: - relevant: false user: 2 So, when calling the program, specify the format as `yaml`. For the query results, a similar format is used. Note that it is necessary to specify whether the result sets are ranked or not, as this will heavily influence the calculations. You can specify the score for a document. By "score" we mean the score that your retrieval algorithm has given the document. But this is not necessary. The documents will always be ranked in the order of their appearance, regardless of their score. Thus in the following example, the document with "07" at the end is the first and "25" is the last, regardless of the score. --- query: 12th air force germany 1957 ranked: true documents: - score: 0.44034874 document: g5701s.ict21307 - score: 0.44034874 document: g5701s.ict21309 - score: 0.44034874 document: g5701s.ict21311 - score: 0.44034874 document: g5701s.ict21313 - score: 0.44034874 document: g5701s.ict21315 - score: 0.44034874 document: g5701s.ict21317 - score: 0.44034874 document: g5701s.ict21319 - score: 0.44034874 document: g5701s.ict21321 - score: 0.44034874 document: g5701s.ict21323 - score: 0.44034874 document: g5701s.ict21325 --- query: 1612 ranked: true documents: - score: 1.0174774 document: g3290.np000144 - score: 0.763108 document: g3201b.ct000726 - score: 0.763108 document: g3400.ct000886 - score: 0.6359234 document: g3201s.ct000130 --- **Note**: You can also use the `plain` format, which will load the gold standard in a different way (but not the results): my_query my_document_1 false my_query my_document_2 true See that every query/document/relevancy pair is separated by a tabulator? You can also add the user's ID in the fourth column if necessary. Running the evaluation ----------------------- After you have specified the input files and the format, you can run the program. If needed, the `-v` switch will turn on verbose messages, such as information on how many judgements, documents and users there are, but this shouldn't be necessary. The program will first load the gold standard and then calculate the statistics for each result set. The output files are automatically created and contain a YAML representation of the results. Calculations may take a while depending on the amount of judgements and documents. If there are a thousand judgements, always consider a few seconds for each result set. Interpreting the output files ------------------------------ Two output files will be created: - `output_avg_precision.yml` - `output_statistics.yml` The first lists the average precision for each query in the query result file. The second file lists all supported statistics for each query in the query results file. For example, for a ranked evaluation, the first two entries of such a query result statistic look like this: --- 12th air force germany 1957: - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 1 :true_negatives: 2516 :true_positives: 0 :document: g5701s.ict21313 :relevant: false - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 2 :true_negatives: 2515 :true_positives: 0 :document: g5701s.ict21317 :relevant: false You can see the precision and recall for that specific point and also the number of documents for the contingency table (true/false positives/negatives). Also, the document identifier is given. API Usage ========= Using this API in another ruby application is probably the more common use case. All you have to do is include the Gem in your Ruby or Ruby on Rails application. For details about available methods, please refer to the API documentation generated by RDoc. **Important**: For this implementation, we use the document ID, the query and the user ID as the primary keys for matching objects. This means that your documents and queries are identified by a string and thus the strings should be sanitized first. Loading the Gold Standard ------------------------- Once you have loaded the Gem, you will probably start by creating a new gold standard. gold_standard = GoldStandard.new Then, you can load judgements into this standard, either from a file, or manually: gold_standard.load_from_yaml_file "my-file.yml" gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John There is a nice shortcut for the `add_judgement` method. Both lines are essentially the same: gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John gold_standard << :document => doc_id, :query => query_string, :relevant => boolean, :user => John Note the usage of typical Rails hashes for better readability (also, this Gem was developed to be used in a Rails webapp). Now that you have loaded the gold standard, you can do things like: gold_standard.contains_judgement? :document => "a document", :query => "the query" gold_standard.relevant? :document => "a document", :query => "the query" Loading the Query Results ------------------------- Now we want to create a new `QueryResultSet`. A query result set can contain more than one result, which is what we normally want. It is important that you specify the gold standard it belongs to. query_result_set = QueryResultSet.new :gold_standard => gold_standard Just like the Gold Standard, you can read a query result set from a file: query_result_set.load_from_yaml_file "my-results-file.yml" Alternatively, you can load the query results one by one. To do this, you have to create the results (either ranked or unranked) and then add documents: my_result = RankedQueryResult.new :query => "the query" my_result.add_document :document => "test_document 1", :score => 13 my_result.add_document :document => "test_document 2", :score => 11 my_result.add_document :document => "test_document 3", :score => 3 This result would be ranked, obviously, and contain three documents. Documents can have a score, but this is optional. You can also create an Array of documents first and add them altogether: documents = Array.new documents << ResultDocument.new :id => "test_document 1", :score => 20 documents << ResultDocument.new :id => "test_document 2", :score => 21 my_result = RankedQueryResult.new :query => "the query", :documents => documents The same applies to `UnrankedQueryResult`s, obviously. The order of ranked documents is the same as the order in which they were added to the result. The `QueryResultSet` will now contain all the results. They are stored in an array called `query_results`, which you can access. So, to iterate over each result, you might want to use the following code: query_result_set.query_results.each_with_index do |result, index| # ... end Or, more simply: for result in query_result_set.query_results # ... end Calculating statistics ---------------------- Now to the interesting part: Calculating statistics. As mentioned before, there is a conceptual difference between ranked and unranked results. Unranked results are much easier to calculate and thus take less CPU time. No matter if unranked or ranked, you can get the most important statistics by just calling the `statistics` method. statistics = my_result.statistics In the simple case of an unranked result, you will receive a hash with the following information: * `precision` - the precision of the results * `recall` - the recall of the results * `false_negatives` - number of not retrieved but relevant items * `false_positives` - number of retrieved but nonrelevant * `true_negatives` - number of not retrieved and nonrelevantv items * `true_positives` - number of retrieved and relevant items In case of a ranked result, you will receive an Array that consists of _n_ such Hashes, depending on the number of documents. Each Hash will give you the information at a certain rank, e.g. the following to lines return the recall at the fourth rank. statistics = my_ranked_result.statistics statistics[3][:recall] In addition to the information mentioned above, you can also get for each rank: * `document` - the ID of the document that was returned at this rank * `relevant` - whether the document was relevant or not Calculating statistics with missing judgements ---------------------------------------------- Sometimes, you don't have judgements for all document/query pairs in the gold standard. If this happens, the results will be cleaned up first. This means that every document in the results that doesn't appear to have a judgement will be removed temporarily. As an example, take the following results: * A * B * C * D Our gold standard only contains judgements for A and C. The results will be cleaned up first, thus leading to: * A * C With this approach, we can still provide meaningful results (for precision and recall). Other statistics ---------------- There are several other statistics that can be calculated, for example the **F measure**. The F measure weighs precision and recall and has one parameter, either "alpha" or "beta". Get the F measure like so: my_result.f_measure :beta => 1 If you don't specify either alpha or beta, we will assume that beta = 1. Another interesting measure is **Cohen's Kappa**, which tells us about the inter-agreement of assessors. Get the kappa statistic like this: gold_standard.kappa This will calculate the average kappa for each pairwise combination of users in the gold standard. For ranked results one might also want to calculate an **11-point precision**. Just call the following: my_ranked_result.eleven_point_precision This will return a Hash that has indices at the 11 recall levels from 0 to 1 (with steps of 0.1) and the corresponding precision at that recall level.

No description provided.

No description provided.

No description provided.