[LangChain] Improve Jupyter Notebooks and add software tests

[amotl]

About

This patch adds software tests to the examples around LangChain, in order to validate that the Jupyter Notebooks work well, and to keep them working / get alerted if something breaks.

Backlog

• The vector search notebook test_notebook[vector_search.ipynb] is yet untested. It will be addressed on behalf of a subsequent iteration. See [LangChain] Fix testing Jupyter Notebook vector_search.ipynb #98.

Details

This patch builds upon GH-96 and GH-85.

/cc @andnig, @marijaselakovic, @karynzv, @hlcianfagna, @hammerhead, @ckurze, @WalBeh

[amotl]

Problem

-DELETE OK, 30 rows affected (0.000 sec)
+DELETE OK, 0 rows affected (0.000 sec)

-- https://github.com/crate/cratedb-examples/actions/runs/6684017889/job/18160822060?pr=97#step:5:585

Source

cratedb-examples/framework/langchain/document_loader.ipynb

Lines 55 to 77 in 1ff62c7

	"outputs": [
	{
	"name": "stdout",
	"output_type": "stream",
	"text": [
	"\u001B[32mCONNECT OK\r\n",
	"\u001B[0m\u001B[32mPSQL OK, 1 row affected (0.001 sec)\r\n",
	"\u001B[0m\u001B[32mDELETE OK, 30 rows affected (0.010 sec)\r\n",
	"\u001B[0m\u001B[32mINSERT OK, 30 rows affected (0.011 sec)\r\n",
	"\u001B[0m\u001B[0m\u001B[32mCONNECT OK\r\n",
	"\u001B[0m\u001B[32mREFRESH OK, 1 row affected (0.026 sec)\r\n",
	"\u001B[0m\u001B[0m"
	]
	}
	],
	"source": [
	"!rm -f mlb_teams_2012.sql\n",
	"!wget --quiet https://github.com/crate-workbench/langchain/raw/cratedb/docs/docs/integrations/document_loaders/example_data/mlb_teams_2012.sql\n",
	"\n",
	"!crash --schema=notebook < mlb_teams_2012.sql;\n",
	"!crash --schema=notebook --command \"REFRESH TABLE mlb_teams_2012;\""
	]
	},

[amotl]

-DELETE OK, 30 rows affected (0.000 sec)
+DELETE OK, 0 rows affected (0.000 sec)

To make the output more idempotent, the SQL DDL statements have been adjusted.

• crate-workbench/langchain@2bf5c60
• crate-workbench/langchain@14d7843

[amotl]

Status Quo

It works. 🚀

collecting ... collected 3 items

test.py::test_notebook[document_loader.ipynb] PASSED
test.py::test_notebook[vector_search.ipynb] SKIPPED (OPENAI_API_KEY not given [skip-notebook])
test.py::test_notebook[conversational_memory.ipynb] PASSED

-- https://github.com/crate/cratedb-examples/actions/runs/6684315548/job/18161437538?pr=97#step:5:519

Outlook

test_notebook[vector_search.ipynb] will be addressed on behalf of a subsequent iteration.

[amotl]

It is a utility temporarily added here. Based on the outcome from chrisjsewell/pytest-notebook#43, it may be added to the upstream library, or to a different spot.

[amotl]

The code has been added to another library, to not clutter the canonical cratedb-examples repository with such utility code.

-- https://github.com/pyveci/pueblo/blob/v0.0.1/pueblo/testing/notebook.py

[amotl]

It is so good to have this tested now. 🚀

Even if the Notebook worked once upon a time, it did not in its current form, because the previous implementation changed.

It is difficult to catch such errors without running rigid software testing, minor things like this easily slip through, effectively rendering the notebook unusable for all people trying to exercise it.

[amotl]

Idempotency flaws when testing. Fixed now.