Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:add missing per-node online documentation
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:62210dee17bfbfb98c67902ecd6b46125fbb3037
User & Date: brandon 2013-11-17 16:39:03
Context
2013-11-17
16:40
fix parenthetical reference check-in: 6cf9b90f00 user: brandon tags: trunk
16:39
add missing per-node online documentation check-in: 62210dee17 user: brandon tags: trunk
15:03
update online docs check-in: 9224571d57 user: brandon tags: trunk, v2.0.2
Changes

Added www/html_node/Back_002dends.html.





















































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).

Copyright (C) 2013  Brandon Invergo

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU
Free Documentation License".

A copy of the license is also available from the Free Software
Foundation Web site at http://www.gnu.org/licenses/fdl.html.


The document was typeset with
http://www.texinfo.org/ (GNU Texinfo).
 -->
<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
<head>
<title>zeptodb: Back-ends</title>

<meta name="description" content="zeptodb: Back-ends">
<meta name="keywords" content="zeptodb: Back-ends">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Index.html#Index" rel="index" title="Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Introduction.html#Introduction" rel="up" title="Introduction">
<link href="Commands.html#Commands" rel="next" title="Commands">
<link href="Tutorial.html#Tutorial" rel="prev" title="Tutorial">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="Back_002dends"></a>
<div class="header">
<p>
Previous: <a href="Tutorial.html#Tutorial" accesskey="p" rel="prev">Tutorial</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Back_002dends-1"></a>
<h3 class="section">1.2 Back-ends</h3>

<p>By default, zeptodb uses the <a href="http://www.gnu.org/software/gdb">GNU dbm</a> (GDBM) library to create and manipulate the DBM databases.
Alternatively, you may choose to use the
<a href="http://fallabs.com/kyotocabinet/">Kyoto Cabinet</a> library
instead.  This is specified by passing the
<samp>--with-kyotocabinet</samp> option to the <samp>configure</samp> script
before compiling zeptodb.
</p>
<p>Note that databases created with these two different back-ends are
<em>not</em> compatible, thus databases created with Kyoto Cabinet can
only be accessed by zeptodb if it has been compiled with support for
the library.
</p>
<p>Databases created with Kyoto Cabinet are required to have the
<samp>.kch</samp> file extension.  By convention, databases created with
GDBM should have the <samp>.db</samp> file extension.
</p>
<p>For most purposes, databases created with GDBM should be sufficient.
For particularly large data sets, however, Kyoto Cabinet is
preferred, since it can add values more quickly and has a much larger
upper limit on the database size.  On the other hand, Kyoto Cabinet is
not as widely available in GNU/Linux distributions as GDBM so it often
must be installed manually.
</p>



</body>
</html>

Added www/html_node/Tutorial.html.













































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).

Copyright (C) 2013  Brandon Invergo

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU
Free Documentation License".

A copy of the license is also available from the Free Software
Foundation Web site at http://www.gnu.org/licenses/fdl.html.


The document was typeset with
http://www.texinfo.org/ (GNU Texinfo).
 -->
<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
<head>
<title>zeptodb: Tutorial</title>

<meta name="description" content="zeptodb: Tutorial">
<meta name="keywords" content="zeptodb: Tutorial">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Index.html#Index" rel="index" title="Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Introduction.html#Introduction" rel="up" title="Introduction">
<link href="Back_002dends.html#Back_002dends" rel="next" title="Back-ends">
<link href="Introduction.html#Introduction" rel="prev" title="Introduction">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="Tutorial"></a>
<div class="header">
<p>
Next: <a href="Back_002dends.html#Back_002dends" accesskey="n" rel="next">Back-ends</a>, Previous: <a href="Introduction.html#Introduction" accesskey="p" rel="prev">Introduction</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Tutorial-1"></a>
<h3 class="section">1.1 Tutorial</h3>

<p>The zeptodb tools are used to create small databases that are stored to
disk and then to store, fetch and remove records from those databases.
Note that these databases are much simpler than, say, SQL databases.
The databases follow the DBM format as created by the GDBM library
see <a href="Back_002dends.html#Back_002dends">Back-ends</a>.  Each record in a DBM database consists of a key and
a value.  All keys and values are stored as plain text, regardless of
their formats.
</p>
<p>First, you create a new database with <code>zdbc</code>:
</p>
<div class="example">
<pre class="example">$ zdbc foo.db
</pre></div>

<p>You can customize the creation of a database in two ways.  The first is
by specifying the number of &quot;buckets&quot; that comprise the database,
specified via the <samp>-b</samp>/<samp>--num-buckets</samp> option.  A DBM
database can be imagined as a series of buckets.  When a new item is
added, an algorithm determines which bucket it belongs in based on its
value.  Likewise, the same algorithm will be used in determining the
bucket from which to fetch an item.  If each bucket only contains a
maximum of one item, then you are guaranteed to be able to find any item
in the same amount of time as any other item.  On the other hand, if the
number of buckets is smaller than the number of items, then when you go
to fetch an item from a bucket, you might then have to search through
all the items in that bucket to find the one that you want.  This might
slow you down.  On the other hand, if the number of buckets is far
greater than the maximum number of items that will be added, the
algorithm will be wasteful.  Thus it&rsquo;s best to use a number of buckets
that will be slightly greater than the expected maximum number of items.
As a rule of them, use about four times more buckets.
</p>
<p>The second option is the size (in bytes) of the memory mapped region to
use, via the <samp>-m</samp>/<samp>--mmap-size</samp> option.  While the
database is stored on the disk as a file, when it is opened by zeptodb,
some or all of that file is mapped in a one-to-one manner with a region
of virtual memory.  Thus, when the program reads from some address in
that region of memory, it reads directly from the corresponding address
in the file.  This will generally speed up reading and writing compared
to traditional file access.  If the memory-mapped region is smaller than
the size of the database, only portions of the file can be mapped at a
time, thus slowing down performance.  Therefore, it is recommended to
use a sufficiently larger value than the size of the database (taking
into account the expected number of records and the size of the data
that is expected to fill the record values).
</p>
<p>Thus, for a big database, you might do:
</p>
<div class="example">
<pre class="example">$ zdbc --num-buckets=10000 --mmap-size=512000000 big.db
</pre></div>

<p>With the database created, you may now store values to it using
<code>zdbs</code>.  <code>zdbs</code> normally takes its input from
<samp>stdin</samp>.  It expects one record per line and for each key/value
pair to be separated by a delimiter character (&rsquo;|&rsquo; by default).  Note
that records are unique: an attempt to store a record with a
pre-existing key will overwrite that record with a new value.
</p>
<p>For example, let&rsquo;s say that you have a text file <samp>emails.txt</samp>
containing the following records:
</p>
<div class="example">
<pre class="example">Brandon|foo@example.com
Joe|bar@example.com
Mary|baz@example.com
</pre></div>

<p>You could store the records in <samp>foo.db</samp> like so:
</p>
<div class="example">
<pre class="example">$ zdbs foo.db &lt;emails.txt
</pre></div>

<p>Note that if you simply don&rsquo;t like shell redirections like this, you can
also use the <samp>-i</samp> or <samp>--input</samp> option to specify the input
file:
</p>
<div class="example">
<pre class="example">$ zdbs -i emails.txt foo.db
</pre></div>

<p>Of course, it&rsquo;s more likely that you&rsquo;ll want to pipe in records from
some other process:
</p>
<div class="example">
<pre class="example">$ fancy_pipeline.sh | zdbs foo.db
</pre></div>

<p>If your records are formatted differently, using, say, &rsquo;-&rsquo; as the
delimiter (i.e &quot;key-value&quot;), you would specify it using the <samp>-d</samp>
or <samp>--delimiter</samp> option.
</p>
<p>Records can then be fetched from the database using <code>zdbf</code>.  In
this case, queries in the form of keys with one key per line are read
from <samp>stdin</samp>:
</p>
<div class="example">
<pre class="example">$ zdbf foo.db
Brandon
foo@example.com
Joe
bar@example.com
Jon
../trunk/src/zdbf: Key does not exist in the database: Jon: Invalid argument
</pre></div>

<p>As with <code>zdbs</code>, you can also specify a file containing the
queries using the <samp>-i</samp> option or you can read them in through a
pipe.
</p>
<p>If you would prefer the output to include the key, you must specify your
desired delimiter using the <samp>-d</samp> option:
</p>
<div class="example">
<pre class="example">$ echo Brandon | zdbf -d':' foo.db
Brandon:foo@example.com
</pre></div>

<p>Finally, you can dump out all of the contents of the database using the
<samp>-a</samp> option.  Note that the order of the records is in no way
guaranteed.
</p>
<div class="example">
<pre class="example">$ zdbf -d'|' -a foo.db
Joe|bar@example.com
Brandon|foo@example.com
Mary|baz@example.com
</pre></div>

<p>Records may then be removed from the database using <code>zdbr</code>.
<code>zdbr</code> operates in a very similar way to <code>zdbf</code>:
</p>
<div class="example">
<pre class="example">$ zdbr foo.db &lt;&lt;EOF
&gt; Brandon
&gt; Joe
&gt; EOF
$ zdbf -a -d'|' foo.db
Mary|baz@example.com
</pre></div>

<hr>
<div class="header">
<p>
Next: <a href="Back_002dends.html#Back_002dends" accesskey="n" rel="next">Back-ends</a>, Previous: <a href="Introduction.html#Introduction" accesskey="p" rel="prev">Introduction</a>, Up: <a href="Introduction.html#Introduction" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
</div>



</body>
</html>

Added www/html_node/zdbc.html.

























































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- This manual is for zeptodb (version 2.0.2, updated 13 November 2013).

Copyright (C) 2013  Brandon Invergo

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU
Free Documentation License".

A copy of the license is also available from the Free Software
Foundation Web site at http://www.gnu.org/licenses/fdl.html.


The document was typeset with
http://www.texinfo.org/ (GNU Texinfo).
 -->
<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
<head>
<title>zeptodb: zdbc</title>

<meta name="description" content="zeptodb: zdbc">
<meta name="keywords" content="zeptodb: zdbc">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Index.html#Index" rel="index" title="Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Commands.html#Commands" rel="up" title="Commands">
<link href="zdbs.html#zdbs" rel="next" title="zdbs">
<link href="Commands.html#Commands" rel="prev" title="Commands">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="zdbc"></a>
<div class="header">
<p>
Next: <a href="zdbs.html#zdbs" accesskey="n" rel="next">zdbs</a>, Previous: <a href="Commands.html#Commands" accesskey="p" rel="prev">Commands</a>, Up: <a href="Commands.html#Commands" accesskey="u" rel="up">Commands</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Index.html#Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="zdbc-1"></a>
<h3 class="section">2.1 zdbc</h3>

<p><code>zdbc</code> is used to create a new database file.  It accepts two
options, one to choose the number of buckets for the database and the
other to choose the size of the memory-mapped region.  These options
may only be set upon database creation and may not be altered later.
</p>
<p>As a general rule of thumb, you should have around one to four times
as many buckets as entries in the database.  So, if your database will
have 200 entries, you should specify 200 to 800 buckets.  A greater
number of buckets lowers the probability of collisions (two entries
mapping to the same location).
</p>
<p>If possible, you should set the size of the memory-mapped region (in
bytes) to be larger than the expected size of the database or
otherwise as large as possible.
</p>
<dl compact="compact">
<dt><samp>-b, --num-buckets=NUM</samp></dt>
<dd><p>The number of buckets to use
</p>
</dd>
<dt><samp>-m, --mmap-size=NUM</samp></dt>
<dd><p>The size (in bytes) of the memory-mapped region to use
</p>
</dd>
<dt><samp>-v, --verbose</samp></dt>
<dd><p>Print more run-time information
</p>
</dd>
<dt><samp>-?, --help</samp></dt>
<dd><p>Show helpful information
</p>
</dd>
<dt><samp>--usage</samp></dt>
<dd><p>Show shorter helpful information
</p>
</dd>
<dt><samp>-V, --version</samp></dt>
<dd><p>Print the program version
</p></dd>
</dl>




</body>
</html>