encode a string to different characters encodings
URL utilities for markdown-it
Small package to encode or decode IP addresses from buffers to strings.
High-performance Base64 encoder and decoder
Encode and decode base64 encoded strings
Encode and decode quoted-printable strings according to rfc2045
Encode/decode base64 data into ArrayBuffers
HTTP server cookie parsing and serialization
utf8 to/from bytes codec (esm/cjs)
Convert character encodings in pure javascript.
Encode a URL to a percent-encoded form, excluding already-encoded sequences
Encode/decode base64 data into ArrayBuffers
```ts import { encode, decode } from 'safe-url-encode' const str = encode('你 好 🤔') //'5L2gIPCfkI4g5Li%2FIOS6hg%3D%3D' const raw = decode(str) // '你 好 🤔' ```
micromark utility to encode dangerous html characters
A wide-character aware text alignment function for use on the console or with fixed width fonts.
base64 utilities for TypeScript and JavaScript
Generate a unique character string suitible for use in files and URLs.
A port of inflection-js to node.js module
Fastest HTML entities encode/decode library.
Match balanced character pairs, like "{" and "}"
TypeScript definitions for strict-uri-encode
base64 to/from bytes codec (esm/cjs)
A stricter URI encode adhering to RFC 3986
Brace expansion as known from sh/bash
:: wmainfo-rb :: Author: Darren Kirby mailto:bulliver@badcomputer.org License: Ruby = Quick API docs = == Initializing == require 'wmainfo' foo = WmaInfo.new("someSong.wma") ... or ... foo = WmaInfo.new("someVideo.wmv", :encoding=>"UTF-16LE") (default encoding is ASCII) ... or ... foo = WmaInfo.new("someVideo.wmv", :debug=>1) == Public attributes == @drm :: 'true' if DRM present else 'false' @tags :: dict of strings (id3 like data) @info :: dict of variable types (non-id3 like data) @ext_info :: dict of variable types (non-id3 like data) from ASF_Extended_Content_Description_Object @headerObject :: dict of arrays (name, GUID, size and offset of ASF objects) @stream :: dict of variable types (stream properties data) == Public methods == print_objects :: pretty-print header objects hasdrm? :: returns True if file has DRM hastag?('str') :: returns True if @tags['str'] exists print_tags :: pretty-print @tags dict hasinfo?('str') :: returns True if @info['str'] exists print_info :: pretty-print @info dict print_stream :: pretty-print @stream dict For more/different documentation see http://badcomputer.org/unix/code/wmainfo/ == Thanks/Contributors == Ilmari Heikkinen sent in a fix for uninitialized '@ext_info'. Guillaume Pierronnet sent in a patch which improves character encoding handling.
:: wmainfo-rb :: Authors: Darren Kirby, Guillaume Pierronnet mailto:bulliver@gmail.com License: Ruby = Quick API docs = == Initializing == require 'wmainfo' foo = WmaInfo.new("someSong.wma") ... or ... foo = WmaInfo.new("someVideo.wmv", :encoding=>"UTF-16LE") (default encoding is ASCII) ... or ... foo = WmaInfo.new("someVideo.wmv", :debug=>1) == Public attributes == @drm :: 'true' if DRM present else 'false' @tags :: dict of strings (id3 like data) @info :: dict of variable types (non-id3 like data) @ext_info :: dict of variable types (non-id3 like data) from ASF_Extended_Content_Description_Object @headerObject :: dict of arrays (name, GUID, size and offset of ASF objects) @stream :: dict of variable types (stream properties data) == Public methods == print_objects :: pretty-print header objects hasdrm? :: returns True if file has DRM hastag?('str') :: returns True if @tags['str'] exists print_tags :: pretty-print @tags dict hasinfo?('str') :: returns True if @info['str'] exists print_info :: pretty-print @info dict print_stream :: pretty-print @stream dict == Thanks/Contributors == Ilmari Heikkinen sent in a fix for uninitialized '@ext_info'. Guillaume Pierronnet sent in a patch which improves character encoding handling.
== ICU4R - ICU Unicode bindings for Ruby ICU4R is an attempt to provide better Unicode support for Ruby, where it lacks for a long time. Current code is mostly rewritten string.c from Ruby 1.8.3. ICU4R is Ruby C-extension binding for ICU library[1] and provides following classes and functionality: * UString: - String-like class with internal UTF16 storage; - UCA rules for UString comparisons (<=>, casecmp); - encoding(codepage) conversion; \ - Unicode normalization; - transliteration, also rule-based; Bunch of locale-sensitive functions: - upcase/downcase; - string collation; \ - string search; - iterators over text line/word/char/sentence breaks; \ - message formatting (number/currency/string/time); - date and number parsing. * URegexp - unicode regular expressions. * UResourceBundle - access to resource bundles, including ICU locale data. * UCalendar - date manipulation and timezone info. * UConverter - codepage conversions API * UCollator - locale-sensitive string comparison == Install and usage > ruby extconf.rb > make && make check > make install Now, in your scripts just require 'icu4r'. To create RDoc, run > sh tools/doc.sh == Requirements To build and use ICU4R you will need GCC and ICU v3.4 libraries[2]. == Differences from Ruby String and Regexp classes === UString vs String 1. UString substring/index methods use UTF16 codeunit indexes, not code points. 2. UString supports most methods from String class. Missing methods are: capitalize, capitalize!, swapcase, swapcase! %, center, ljust, rjust chomp, chomp!, chop, chop! \ count, delete, delete!, squeeze, squeeze!, tr, tr!, tr_s, tr_s! crypt, intern, sum, unpack dump, each_byte, each_line hex, oct, to_i, to_sym reverse, reverse! succ, succ!, next, next!, upto 3. Instead of String#% method, UString#format is provided. See FORMATTING for short reference. 4. UStrings can be created via String.to_u(encoding='utf8') or global u(str,[encoding='utf8']) calls. Note that +encoding+ parameter must be value of String class. 5. There's difference between character grapheme, codepoint and codeunit. See UNICODE reports for gory details, but in short: locale dependent notion of character can be presented using more than one codepoint - base letter and combining (accents) (also possible more than one!), and each codepoint can require more than one codeunit to store (for UTF8 codeunit size is 8bit, though \ some codepoints require up to 4bytes). So, UString has normalization and locale dependent break iterators. 6. Currently UString doesn't include Enumerable module. 7. UString index/[] methods which accept URegexp, throw exception if Regexp passed. 8. UString#<=>, UString#casecmp use UCA rules. === URegexp UString uses ICU regexp library. Pattern syntax is described in [./docs/UNICODE_REGEXPS] and ICU docs. There are some differences between processing in Ruby Regexp and URegexp: 1. When UString#sub, UString#gsub are called with block, special vars ($~, $&, $1, ...) aren't set, as their values are processed through deep ruby core code. Instead, block receives UMatch object, which is essentially immutable array of matching groups: "test".u.gsub(ure("(e)(.)")) do |match| \ puts match[0] # => 'es' <--> $& puts match[1] # => 'e' \ <--> $1 puts match[2] # => 's' <--> $2 end 2. In URegexp search pattern backreferences are in form \n (\1, \2, ...), in replacement string - in form $1, $2, ... NOTE: URegexp considers char to be a digit NOT ONLY ASCII (0x0030-0x0039), but any Unicode char, which has property Decimal digit number (Nd), e.g.: a = [?$, 0x1D7D9].pack("U*").u * 2 puts a.inspect_names <U000024>DOLLAR SIGN <U01D7D9>MATHEMATICAL DOUBLE-STRUCK DIGIT ONE <U000024>DOLLAR SIGN <U01D7D9>MATHEMATICAL DOUBLE-STRUCK DIGIT ONE puts "abracadabra".u.gsub(/(b)/.U, a) abbracadabbra \ 3. One can create URegexp using global Kernel#ure function, Regexp#U, Regexp#to_u, or from UString using URegexp.new, e.g: /pattern/.U =~ "string".u 4. There are differences about Regexp and URegexp multiline matching options: t = "text\ntest" # ^,$ handling : URegexp multiline <-> Ruby default t.u =~ ure('^\w+$', URegexp::MULTILINE) => #<UMatch:0xf6f7de04 @ranges=[0..3], @cg=[\u0074\u0065\u0078\u0074]> t =~ /^\w+$/ => 0 # . matches \n : URegexp DOTALL <-> /m t.u =~ ure('.+test', URegexp::DOTALL) \ => #<UMatch:0xf6fa4d88 ... t.u =~ /.+test/m 5. UMatch.range(idx) returns range for capturing group idx. This range is in codeunits. === References 1. ICU Official Homepage http://ibm.com/software/globalization/icu/ 2. ICU downloads \ http://ibm.com/software/globalization/icu/downloads.jsp 3. ICU Home Page http://icu.sf.net 4. Unicode Home Page http://www.unicode.org ==== BUGS, DOCS, TO DO The code is slow and inefficient yet, is still highly experimental, so can have many security and memory leaks, bugs, inconsistent documentation, incomplete test suite. Use it at your own risk. Bug reports and feature requests are welcome :) === Copying This extension module is copyrighted free software by Nikolai Lugovoi. You can redistribute it and/or modify it under the terms of MIT License. Nikolai Lugovoi <meadow.nnick@gmail.com>