Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,23 @@ space, as in `machine learning:0.35`.
Run `keywords --help` for the full option list. A usage error exits 2 and any
other error exits 1, so scripts can tell the two apart.

[keywords reference →](docs/keywords.md) · [CLI Guide →](https://rubyclassifier.com/docs/guides/cli/basics)
Run the two commands side by side to read a label together with the terms that
make the text distinctive:

```bash
classifier -f reviews-model.json -p "Broken on arrival, awful quality"
# => positive:0.12 negative:0.88

keywords -m reviews.json -n 5 "Broken on arrival, awful quality"
# => awful:0.52 arrival:0.52 broken:0.52 quality:0.44
```

They keep separate models in separate formats, so `classifier` takes `-f` and
`keywords` takes `-m`, and neither reads the other's file. The terms are
context, not an explanation of the label: TF-IDF measures how well a term
separates a document from its corpus, not how much it favors a category.

[Using both commands →](docs/cli.md#using-both-commands-together) · [keywords reference →](docs/keywords.md) · [CLI Guide →](https://rubyclassifier.com/docs/guides/cli/basics)

### Claude Code Plugin

Expand Down
151 changes: 151 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,157 @@ $ classifier -m lsi related article.txt
| `-v`, `--version` | Print the gem version |
| `-h`, `--help` | Print the full usage |

## Using both commands together

`classifier` and `keywords` answer different questions about the same text.
`classifier` gives a category. `keywords` names the terms that make the text
distinctive against its corpus. Run them side by side to read a label together
with what the document is about.

Fit a vocabulary from the same corpus you train on, and the two views line up:

```console
$ keywords fit -m reviews.json reviews/good/*.txt reviews/bad/*.txt
$ keywords info -m reviews.json
Documents: 8
Vocabulary: 37
Min DF: 1
Max DF: 1.0
```

`keywords info` reports the corpus before you commit to training. A vocabulary
of 37 terms over 8 documents says the corpus is far too small, and no
classifier fixes that.

Then train, and read the two answers together:

```console
$ classifier -f reviews-model.json train positive reviews/good/*.txt
$ classifier -f reviews-model.json train negative reviews/bad/*.txt

$ classifier -f reviews-model.json -p "Broken on arrival, awful quality and useless customer service"
positive:0.07 negative:0.93

$ keywords -m reviews.json -n 5 "Broken on arrival, awful quality and useless customer service"
useless:0.4 awful:0.4 arrival:0.4 broken:0.4 service:0.34
```

The first line is the verdict. The second is the document in shorthand, which
tells you what the classifier was reading when a result surprises you.

**The second line is not an explanation of the first.** The two commands hold
separate models. `keywords` reports TF-IDF weight, which measures how well a
term separates this document from the rest of the corpus. It never sees the
classifier, and it does not know which category a term favors. A term can top
the list and carry no weight in the decision.

Read it as context, not as attribution. When a label looks wrong, the terms
tell you whether the document says what you assumed, which is usually the real
problem. For the weights a model actually holds, use
`Classifier::LogisticRegression#weights` from Ruby, which returns the learned
weight per term and per category:

```ruby
classifier.weights("positive", limit: 5)
```

No command line flag reports per-term weights for a Bayes model.

### Find the words your corpus wastes on itself

A term that appears in nearly every document tells a classifier nothing, and
every corpus grows its own. A review corpus repeats `delivery`, a support
corpus repeats `ticket`. These are stopwords that no general stopword list
knows about, because they are specific to your data.

`keywords` finds them, because `--max-df` drops a term that appears in more
than the given ratio of documents. Compare the vocabulary size before and
after:

```console
$ keywords fit -m default.json good.txt bad.txt
$ keywords info -m default.json
Documents: 12
Vocabulary: 42

$ keywords fit -m pruned.json --max-df 0.5 good.txt bad.txt
$ keywords info -m pruned.json
Documents: 12
Vocabulary: 41
```

One term went. Score a document under each model to see which:

```console
$ keywords -m default.json -n 3 "delivery was awful and broken"
broken:0.69 awful:0.69 delivery:0.24

$ keywords -m pruned.json -n 3 "delivery was awful and broken"
broken:0.71 awful:0.71
```

`delivery` sat in all 12 documents and still drew weight. Dropping it sharpens
every term that carries real signal.

Watch the vocabulary count as you tune, because these bounds cut fast:

```console
$ keywords fit -m tight.json --min-df 2 good.txt bad.txt
$ keywords info -m tight.json
Documents: 12
Vocabulary: 7
```

`--min-df 2` took 42 terms down to 7. That is no longer a vocabulary, it is a
handful of words. Move one bound at a time and read `keywords info` after each
change.

### Two commands, two models

The models are separate files in separate formats, and neither command reads
the other's:

```console
$ classifier -f reviews.json "broken awful"
Error: Unknown classifier type in model: tfidf

$ keywords -m reviews-model.json "broken awful"
Error: Invalid vectorizer type: bayes
```

Note the flags differ too. `classifier` takes `-f`, and `keywords` takes `-m`.

| | `classifier` | `keywords` |
|:--|:--|:--|
| Answers | Which category | Which terms matter |
| Model flag | `-f` | `-m` |
| Default model | `./classifier.json` | `./keywords.json` |
| Builds a model with | `train` | `fit` |
| Pre-trained models | Yes, through `-r` | No |

### Do not pipe one into the other

`keywords` prints `term:score` pairs, which is not text to classify. Feeding
its output to `classifier` throws away the rest of the document and weakens the
result:

```console
$ classifier -f reviews-model.json -p "$LONG_REVIEW"
positive:0.08 negative:0.92 # the whole review

$ keywords -m reviews.json -n 4 "$LONG_REVIEW"
arrived:0.6 refund:0.3 useless:0.3 build:0.3

$ classifier -f reviews-model.json -p "arrived refund useless build"
positive:0.21 negative:0.79 # weaker, from the top terms alone
```

Confidence drops from 0.92 to 0.79. TF-IDF ranks a term by how much it
distinguishes one document from the rest of the corpus, which is not the same
as how much it signals a category. Here it puts `arrived` first, a neutral
word about delivery. Classify the full text, and read `keywords` alongside it
for context rather than for attribution.

## Install without Ruby

Homebrew installs the command line tools on their own:
Expand Down
Loading