Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:add missing per-node online documentation
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:62210dee17bfbfb98c67902ecd6b46125fbb3037
User & Date: brandon 2013-11-17 16:39:03
Context
2013-11-17
16:40
fix parenthetical reference check-in: 6cf9b90f00 user: brandon tags: trunk
16:39
add missing per-node online documentation check-in: 62210dee17 user: brandon tags: trunk
15:03
update online docs check-in: 9224571d57 user: brandon tags: trunk, v2.0.2
Changes

Added www/html_node/Back_002dends.html.

            1  +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
            2  +<html>
            3  +<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).
            4  +
            5  +Copyright (C) 2013  Brandon Invergo
            6  +
            7  +Permission is granted to copy, distribute and/or modify this document
            8  +under the terms of the GNU Free Documentation License, Version 1.3
            9  +or any later version published by the Free Software Foundation;
           10  +with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
           11  +A copy of the license is included in the section entitled "GNU
           12  +Free Documentation License".
           13  +
           14  +A copy of the license is also available from the Free Software
           15  +Foundation Web site at http://www.gnu.org/licenses/fdl.html.
           16  +
           17  +
           18  +The document was typeset with
           19  +http://www.texinfo.org/ (GNU Texinfo).
           20  + -->
           21  +<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
           22  +<head>
           23  +<title>zeptodb: Back-ends</title>
           24  +
           25  +<meta name="description" content="zeptodb: Back-ends">
           26  +<meta name="keywords" content="zeptodb: Back-ends">
           27  +<meta name="resource-type" content="document">
           28  +<meta name="distribution" content="global">
           29  +<meta name="Generator" content="makeinfo">
           30  +<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
           31  +<link href="index.html#Top" rel="start" title="Top">
           32  +<link href="Index.html#Index" rel="index" title="Index">
           33  +<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
           34  +<link href="Introduction.html#Introduction" rel="up" title="Introduction">
           35  +<link href="Commands.html#Commands" rel="next" title="Commands">
           36  +<link href="Tutorial.html#Tutorial" rel="prev" title="Tutorial">
           37  +<style type="text/css">
           38  +<!--
           39  +a.summary-letter {text-decoration: none}
           40  +blockquote.smallquotation {font-size: smaller}
           41  +div.display {margin-left: 3.2em}
           42  +div.example {margin-left: 3.2em}
           43  +div.indentedblock {margin-left: 3.2em}
           44  +div.lisp {margin-left: 3.2em}
           45  +div.smalldisplay {margin-left: 3.2em}
           46  +div.smallexample {margin-left: 3.2em}
           47  +div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
           48  +div.smalllisp {margin-left: 3.2em}
           49  +kbd {font-style:oblique}
           50  +pre.display {font-family: inherit}
           51  +pre.format {font-family: inherit}
           52  +pre.menu-comment {font-family: serif}
           53  +pre.menu-preformatted {font-family: serif}
           54  +pre.smalldisplay {font-family: inherit; font-size: smaller}
           55  +pre.smallexample {font-size: smaller}
           56  +pre.smallformat {font-family: inherit; font-size: smaller}
           57  +pre.smalllisp {font-size: smaller}
           58  +span.nocodebreak {white-space:nowrap}
           59  +span.nolinebreak {white-space:nowrap}
           60  +span.roman {font-family:serif; font-weight:normal}
           61  +span.sansserif {font-family:sans-serif; font-weight:normal}
           62  +ul.no-bullet {list-style: none}
           63  +-->
           64  +</style>
           65  +
           66  +
           67  +</head>
           68  +
           69  +<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
           70  +<a name="Back_002dends"></a>
           71  +<div class="header">
           72  +<p>
           73  +Previous: <a href="Tutorial.html#Tutorial" accesskey="p" rel="prev">Tutorial</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
           74  +</div>
           75  +<hr>
           76  +<a name="Back_002dends-1"></a>
           77  +<h3 class="section">1.2 Back-ends</h3>
           78  +
           79  +<p>By default, zeptodb uses the <a href="http://www.gnu.org/software/gdb">GNU dbm</a> (GDBM) library to create and manipulate the DBM databases.
           80  +Alternatively, you may choose to use the
           81  +<a href="http://fallabs.com/kyotocabinet/">Kyoto Cabinet</a> library
           82  +instead.  This is specified by passing the
           83  +<samp>--with-kyotocabinet</samp> option to the <samp>configure</samp> script
           84  +before compiling zeptodb.
           85  +</p>
           86  +<p>Note that databases created with these two different back-ends are
           87  +<em>not</em> compatible, thus databases created with Kyoto Cabinet can
           88  +only be accessed by zeptodb if it has been compiled with support for
           89  +the library.
           90  +</p>
           91  +<p>Databases created with Kyoto Cabinet are required to have the
           92  +<samp>.kch</samp> file extension.  By convention, databases created with
           93  +GDBM should have the <samp>.db</samp> file extension.
           94  +</p>
           95  +<p>For most purposes, databases created with GDBM should be sufficient.
           96  +For particularly large data sets, however, Kyoto Cabinet is
           97  +preferred, since it can add values more quickly and has a much larger
           98  +upper limit on the database size.  On the other hand, Kyoto Cabinet is
           99  +not as widely available in GNU/Linux distributions as GDBM so it often
          100  +must be installed manually.
          101  +</p>
          102  +
          103  +
          104  +
          105  +</body>
          106  +</html>

Added www/html_node/Tutorial.html.

            1  +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
            2  +<html>
            3  +<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).
            4  +
            5  +Copyright (C) 2013  Brandon Invergo
            6  +
            7  +Permission is granted to copy, distribute and/or modify this document
            8  +under the terms of the GNU Free Documentation License, Version 1.3
            9  +or any later version published by the Free Software Foundation;
           10  +with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
           11  +A copy of the license is included in the section entitled "GNU
           12  +Free Documentation License".
           13  +
           14  +A copy of the license is also available from the Free Software
           15  +Foundation Web site at http://www.gnu.org/licenses/fdl.html.
           16  +
           17  +
           18  +The document was typeset with
           19  +http://www.texinfo.org/ (GNU Texinfo).
           20  + -->
           21  +<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
           22  +<head>
           23  +<title>zeptodb: Tutorial</title>
           24  +
           25  +<meta name="description" content="zeptodb: Tutorial">
           26  +<meta name="keywords" content="zeptodb: Tutorial">
           27  +<meta name="resource-type" content="document">
           28  +<meta name="distribution" content="global">
           29  +<meta name="Generator" content="makeinfo">
           30  +<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
           31  +<link href="index.html#Top" rel="start" title="Top">
           32  +<link href="Index.html#Index" rel="index" title="Index">
           33  +<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
           34  +<link href="Introduction.html#Introduction" rel="up" title="Introduction">
           35  +<link href="Back_002dends.html#Back_002dends" rel="next" title="Back-ends">
           36  +<link href="Introduction.html#Introduction" rel="prev" title="Introduction">
           37  +<style type="text/css">
           38  +<!--
           39  +a.summary-letter {text-decoration: none}
           40  +blockquote.smallquotation {font-size: smaller}
           41  +div.display {margin-left: 3.2em}
           42  +div.example {margin-left: 3.2em}
           43  +div.indentedblock {margin-left: 3.2em}
           44  +div.lisp {margin-left: 3.2em}
           45  +div.smalldisplay {margin-left: 3.2em}
           46  +div.smallexample {margin-left: 3.2em}
           47  +div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
           48  +div.smalllisp {margin-left: 3.2em}
           49  +kbd {font-style:oblique}
           50  +pre.display {font-family: inherit}
           51  +pre.format {font-family: inherit}
           52  +pre.menu-comment {font-family: serif}
           53  +pre.menu-preformatted {font-family: serif}
           54  +pre.smalldisplay {font-family: inherit; font-size: smaller}
           55  +pre.smallexample {font-size: smaller}
           56  +pre.smallformat {font-family: inherit; font-size: smaller}
           57  +pre.smalllisp {font-size: smaller}
           58  +span.nocodebreak {white-space:nowrap}
           59  +span.nolinebreak {white-space:nowrap}
           60  +span.roman {font-family:serif; font-weight:normal}
           61  +span.sansserif {font-family:sans-serif; font-weight:normal}
           62  +ul.no-bullet {list-style: none}
           63  +-->
           64  +</style>
           65  +
           66  +
           67  +</head>
           68  +
           69  +<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
           70  +<a name="Tutorial"></a>
           71  +<div class="header">
           72  +<p>
           73  +Next: <a href="Back_002dends.html#Back_002dends" accesskey="n" rel="next">Back-ends</a>, Previous: <a href="Introduction.html#Introduction" accesskey="p" rel="prev">Introduction</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
           74  +</div>
           75  +<hr>
           76  +<a name="Tutorial-1"></a>
           77  +<h3 class="section">1.1 Tutorial</h3>
           78  +
           79  +<p>The zeptodb tools are used to create small databases that are stored to
           80  +disk and then to store, fetch and remove records from those databases.
           81  +Note that these databases are much simpler than, say, SQL databases.
           82  +The databases follow the DBM format as created by the GDBM library
           83  +see <a href="Back_002dends.html#Back_002dends">Back-ends</a>.  Each record in a DBM database consists of a key and
           84  +a value.  All keys and values are stored as plain text, regardless of
           85  +their formats.
           86  +</p>
           87  +<p>First, you create a new database with <code>zdbc</code>:
           88  +</p>
           89  +<div class="example">
           90  +<pre class="example">$ zdbc foo.db
           91  +</pre></div>
           92  +
           93  +<p>You can customize the creation of a database in two ways.  The first is
           94  +by specifying the number of &quot;buckets&quot; that comprise the database,
           95  +specified via the <samp>-b</samp>/<samp>--num-buckets</samp> option.  A DBM
           96  +database can be imagined as a series of buckets.  When a new item is
           97  +added, an algorithm determines which bucket it belongs in based on its
           98  +value.  Likewise, the same algorithm will be used in determining the
           99  +bucket from which to fetch an item.  If each bucket only contains a
          100  +maximum of one item, then you are guaranteed to be able to find any item
          101  +in the same amount of time as any other item.  On the other hand, if the
          102  +number of buckets is smaller than the number of items, then when you go
          103  +to fetch an item from a bucket, you might then have to search through
          104  +all the items in that bucket to find the one that you want.  This might
          105  +slow you down.  On the other hand, if the number of buckets is far
          106  +greater than the maximum number of items that will be added, the
          107  +algorithm will be wasteful.  Thus it&rsquo;s best to use a number of buckets
          108  +that will be slightly greater than the expected maximum number of items.
          109  +As a rule of them, use about four times more buckets.
          110  +</p>
          111  +<p>The second option is the size (in bytes) of the memory mapped region to
          112  +use, via the <samp>-m</samp>/<samp>--mmap-size</samp> option.  While the
          113  +database is stored on the disk as a file, when it is opened by zeptodb,
          114  +some or all of that file is mapped in a one-to-one manner with a region
          115  +of virtual memory.  Thus, when the program reads from some address in
          116  +that region of memory, it reads directly from the corresponding address
          117  +in the file.  This will generally speed up reading and writing compared
          118  +to traditional file access.  If the memory-mapped region is smaller than
          119  +the size of the database, only portions of the file can be mapped at a
          120  +time, thus slowing down performance.  Therefore, it is recommended to
          121  +use a sufficiently larger value than the size of the database (taking
          122  +into account the expected number of records and the size of the data
          123  +that is expected to fill the record values).
          124  +</p>
          125  +<p>Thus, for a big database, you might do:
          126  +</p>
          127  +<div class="example">
          128  +<pre class="example">$ zdbc --num-buckets=10000 --mmap-size=512000000 big.db
          129  +</pre></div>
          130  +
          131  +<p>With the database created, you may now store values to it using
          132  +<code>zdbs</code>.  <code>zdbs</code> normally takes its input from
          133  +<samp>stdin</samp>.  It expects one record per line and for each key/value
          134  +pair to be separated by a delimiter character (&rsquo;|&rsquo; by default).  Note
          135  +that records are unique: an attempt to store a record with a
          136  +pre-existing key will overwrite that record with a new value.
          137  +</p>
          138  +<p>For example, let&rsquo;s say that you have a text file <samp>emails.txt</samp>
          139  +containing the following records:
          140  +</p>
          141  +<div class="example">
          142  +<pre class="example">Brandon|foo@example.com
          143  +Joe|bar@example.com
          144  +Mary|baz@example.com
          145  +</pre></div>
          146  +
          147  +<p>You could store the records in <samp>foo.db</samp> like so:
          148  +</p>
          149  +<div class="example">
          150  +<pre class="example">$ zdbs foo.db &lt;emails.txt
          151  +</pre></div>
          152  +
          153  +<p>Note that if you simply don&rsquo;t like shell redirections like this, you can
          154  +also use the <samp>-i</samp> or <samp>--input</samp> option to specify the input
          155  +file:
          156  +</p>
          157  +<div class="example">
          158  +<pre class="example">$ zdbs -i emails.txt foo.db
          159  +</pre></div>
          160  +
          161  +<p>Of course, it&rsquo;s more likely that you&rsquo;ll want to pipe in records from
          162  +some other process:
          163  +</p>
          164  +<div class="example">
          165  +<pre class="example">$ fancy_pipeline.sh | zdbs foo.db
          166  +</pre></div>
          167  +
          168  +<p>If your records are formatted differently, using, say, &rsquo;-&rsquo; as the
          169  +delimiter (i.e &quot;key-value&quot;), you would specify it using the <samp>-d</samp>
          170  +or <samp>--delimiter</samp> option.
          171  +</p>
          172  +<p>Records can then be fetched from the database using <code>zdbf</code>.  In
          173  +this case, queries in the form of keys with one key per line are read
          174  +from <samp>stdin</samp>:
          175  +</p>
          176  +<div class="example">
          177  +<pre class="example">$ zdbf foo.db
          178  +Brandon
          179  +foo@example.com
          180  +Joe
          181  +bar@example.com
          182  +Jon
          183  +../trunk/src/zdbf: Key does not exist in the database: Jon: Invalid argument
          184  +</pre></div>
          185  +
          186  +<p>As with <code>zdbs</code>, you can also specify a file containing the
          187  +queries using the <samp>-i</samp> option or you can read them in through a
          188  +pipe.
          189  +</p>
          190  +<p>If you would prefer the output to include the key, you must specify your
          191  +desired delimiter using the <samp>-d</samp> option:
          192  +</p>
          193  +<div class="example">
          194  +<pre class="example">$ echo Brandon | zdbf -d':' foo.db
          195  +Brandon:foo@example.com
          196  +</pre></div>
          197  +
          198  +<p>Finally, you can dump out all of the contents of the database using the
          199  +<samp>-a</samp> option.  Note that the order of the records is in no way
          200  +guaranteed.
          201  +</p>
          202  +<div class="example">
          203  +<pre class="example">$ zdbf -d'|' -a foo.db
          204  +Joe|bar@example.com
          205  +Brandon|foo@example.com
          206  +Mary|baz@example.com
          207  +</pre></div>
          208  +
          209  +<p>Records may then be removed from the database using <code>zdbr</code>.
          210  +<code>zdbr</code> operates in a very similar way to <code>zdbf</code>:
          211  +</p>
          212  +<div class="example">
          213  +<pre class="example">$ zdbr foo.db &lt;&lt;EOF
          214  +&gt; Brandon
          215  +&gt; Joe
          216  +&gt; EOF
          217  +$ zdbf -a -d'|' foo.db
          218  +Mary|baz@example.com
          219  +</pre></div>
          220  +
          221  +<hr>
          222  +<div class="header">
          223  +<p>
          224  +Next: <a href="Back_002dends.html#Back_002dends" accesskey="n" rel="next">Back-ends</a>, Previous: <a href="Introduction.html#Introduction" accesskey="p" rel="prev">Introduction</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
          225  +</div>
          226  +
          227  +
          228  +
          229  +</body>
          230  +</html>

Added www/html_node/zdbc.html.

            1  +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
            2  +<html>
            3  +<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).
            4  +
            5  +Copyright (C) 2013  Brandon Invergo
            6  +
            7  +Permission is granted to copy, distribute and/or modify this document
            8  +under the terms of the GNU Free Documentation License, Version 1.3
            9  +or any later version published by the Free Software Foundation;
           10  +with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
           11  +A copy of the license is included in the section entitled "GNU
           12  +Free Documentation License".
           13  +
           14  +A copy of the license is also available from the Free Software
           15  +Foundation Web site at http://www.gnu.org/licenses/fdl.html.
           16  +
           17  +
           18  +The document was typeset with
           19  +http://www.texinfo.org/ (GNU Texinfo).
           20  + -->
           21  +<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
           22  +<head>
           23  +<title>zeptodb: zdbc</title>
           24  +
           25  +<meta name="description" content="zeptodb: zdbc">
           26  +<meta name="keywords" content="zeptodb: zdbc">
           27  +<meta name="resource-type" content="document">
           28  +<meta name="distribution" content="global">
           29  +<meta name="Generator" content="makeinfo">
           30  +<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
           31  +<link href="index.html#Top" rel="start" title="Top">
           32  +<link href="Index.html#Index" rel="index" title="Index">
           33  +<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
           34  +<link href="Commands.html#Commands" rel="up" title="Commands">
           35  +<link href="zdbs.html#zdbs" rel="next" title="zdbs">
           36  +<link href="Commands.html#Commands" rel="prev" title="Commands">
           37  +<style type="text/css">
           38  +<!--
           39  +a.summary-letter {text-decoration: none}
           40  +blockquote.smallquotation {font-size: smaller}
           41  +div.display {margin-left: 3.2em}
           42  +div.example {margin-left: 3.2em}
           43  +div.indentedblock {margin-left: 3.2em}
           44  +div.lisp {margin-left: 3.2em}
           45  +div.smalldisplay {margin-left: 3.2em}
           46  +div.smallexample {margin-left: 3.2em}
           47  +div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
           48  +div.smalllisp {margin-left: 3.2em}
           49  +kbd {font-style:oblique}
           50  +pre.display {font-family: inherit}
           51  +pre.format {font-family: inherit}
           52  +pre.menu-comment {font-family: serif}
           53  +pre.menu-preformatted {font-family: serif}
           54  +pre.smalldisplay {font-family: inherit; font-size: smaller}
           55  +pre.smallexample {font-size: smaller}
           56  +pre.smallformat {font-family: inherit; font-size: smaller}
           57  +pre.smalllisp {font-size: smaller}
           58  +span.nocodebreak {white-space:nowrap}
           59  +span.nolinebreak {white-space:nowrap}
           60  +span.roman {font-family:serif; font-weight:normal}
           61  +span.sansserif {font-family:sans-serif; font-weight:normal}
           62  +ul.no-bullet {list-style: none}
           63  +-->
           64  +</style>
           65  +
           66  +
           67  +</head>
           68  +
           69  +<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
           70  +<a name="zdbc"></a>
           71  +<div class="header">
           72  +<p>
           73  +Next: <a href="zdbs.html#zdbs" accesskey="n" rel="next">zdbs</a>, Previous: <a href="Commands.html#Commands" accesskey="p" rel="prev">Commands</a>, Up: <a href="Commands.html#Commands" accesskey="u" rel="up">Commands</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
           74  +</div>
           75  +<hr>
           76  +<a name="zdbc-1"></a>
           77  +<h3 class="section">2.1 zdbc</h3>
           78  +
           79  +<p><code>zdbc</code> is used to create a new database file.  It accepts two
           80  +options, one to choose the number of buckets for the database and the
           81  +other to choose the size of the memory-mapped region.  These options
           82  +may only be set upon database creation and may not be altered later.
           83  +</p>
           84  +<p>As a general rule of thumb, you should have around one to four times
           85  +as many buckets as entries in the database.  So, if your database will
           86  +have 200 entries, you should specify 200 to 800 buckets.  A greater
           87  +number of buckets lowers the probability of collisions (two entries
           88  +mapping to the same location).
           89  +</p>
           90  +<p>If possible, you should set the size of the memory-mapped region (in
           91  +bytes) to be larger than the expected size of the database or
           92  +otherwise as large as possible.
           93  +</p>
           94  +<dl compact="compact">
           95  +<dt><samp>-b, --num-buckets=NUM</samp></dt>
           96  +<dd><p>The number of buckets to use
           97  +</p>
           98  +</dd>
           99  +<dt><samp>-m, --mmap-size=NUM</samp></dt>
          100  +<dd><p>The size (in bytes) of the memory-mapped region to use
          101  +</p>
          102  +</dd>
          103  +<dt><samp>-v, --verbose</samp></dt>
          104  +<dd><p>Print more run-time information
          105  +</p>
          106  +</dd>
          107  +<dt><samp>-?, --help</samp></dt>
          108  +<dd><p>Show helpful information
          109  +</p>
          110  +</dd>
          111  +<dt><samp>--usage</samp></dt>
          112  +<dd><p>Show shorter helpful information
          113  +</p>
          114  +</dd>
          115  +<dt><samp>-V, --version</samp></dt>
          116  +<dd><p>Print the program version
          117  +</p></dd>
          118  +</dl>
          119  +
          120  +
          121  +
          122  +
          123  +</body>
          124  +</html>