Skip to content

Implement docs with sphinx #508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 4, 2025
Merged

Implement docs with sphinx #508

merged 8 commits into from
Jun 4, 2025

Conversation

jhamon
Copy link
Collaborator

@jhamon jhamon commented Jun 3, 2025

Problem

Recent changes to implement lazy loading in the SDK have broken compatibility with our previous docs tool, pdoc. After some investigation, it seemed easier to adopt a new docs solution than to try to manipulate pdoc internals to work for our usecase.

Solution

  • Implement configurations for docs in a new docs folder. Main points of interest: docs/conf.py and docs/index.rst
  • Adjust docstrings to use rst format throughout
  • Add a few custom styles in docs/_static/custom.css to tweak the look of the default sphinx theme which is using Garamond font and doesn't really match our branding at all. Changing up to use sans-serif fonts and tweaking a few other things gets us a long way there, although I'm sure there are still things to refine on that over time.
  • Build docs with poetry run sphinx-build -b html docs docsbuild
  • Implement some minor shenanigans to work around poetry and sphinx's rigidness about python version support. To do this in a nicer way I would either have to drop python 3.9 and 3.10 from our SDK's supported python versions, or setup a separate package outside this repo that depended on sphinx and the published pinecone. Neither one of those sound like good options to me.

Type of Change

  • New feature (non-breaking change which adds functionality)

Testing

New docs visible at https://sdk.pinecone.io/python

I published these by manually triggering the build-and-publish-docs job from this branch

@@ -0,0 +1,260 @@
.. toctree::
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is the main entry point for sphinx when the static site is regenerated.

The .. toctree:: directive is how navigation is defined and dictates what other content is pulled in.

@@ -0,0 +1,126 @@
========
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To determine what methods are actually documented, they need to be declared in an rst file that is mentioned in the root toctree. Currently we have three things we're documenting: Pinecone in rest.rst, PineconeAsyncio in asyncio.rst, and PineconeGRPC in grpc.rst.

Comment on lines -1 to -4
> [!NOTE]
> The official SDK package was renamed from `pinecone-client` to `pinecone` beginning in version 5.1.0.
> Please remove `pinecone-client` from your project dependencies and add `pinecone` instead to get
> the latest updates.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using something called myst-parser to suck in some content that already existed in the repo as markdown files and render it into the sphinx static site. For the most part it does a good job, but the myst-parser didn't handle this comment block syntax. I think this is a github-specific addition to the markdown syntax, so I guess I'm not surprised.

@@ -1,62 +0,0 @@
class IndexClientInstantiationError(Exception):
Copy link
Collaborator Author

@jhamon jhamon Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the past we were doing some weird stuff where we aliased the real implementation of Index to _Index and then threw an error message if anybody tried to directly instantiate the Index class. I can't remember now, but I think this was something a field engineer requested.

I decided to nix these because it was causing a problem when generating docs. In the past, this was added to chastise people if they tried to instantiate the Index class without going through a parent client object such as Pinecone (which was handling a bunch of configuration stuff), but that small benefit is not worth having wonky looking documentation.

@@ -30,6 +30,7 @@ jobs:
if: ${{ always() }}
needs:
- dependency-tests
- create-project
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I removed pdoc, this workflow got triggered and I noticed this mistake in my recent "create a project for every test run" PR.

Comment on lines +19 to +32
- name: Pretend this project requires Python 3.11
shell: bash
run: |
# Poetry won't let me install sphinx as a dev dependency in this project
# because of the wide range of versions our library supports. So during this
# action, we'll pretend this project requires Python 3.11 or greater.
sed -i 's/python = "^3.9"/python = "^3.11"/' pyproject.toml
poetry lock
poetry install -E grpc -E asyncio

- name: Install sphinx
shell: bash
run: |
poetry add sphinx myst-parser --group dev
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a minor atrocity, but for now it's the best way to work around sphinx's relatively narrow python version support. Poetry won't let me install it normally (and add it as a dev dependency in pyproject.toml) because our library has a wider range of python version support than sphinx.

@jhamon jhamon marked this pull request as ready for review June 4, 2025 20:05
Copy link
Contributor

@rohanshah18 rohanshah18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, LGTM!

@jhamon jhamon merged commit d1282ce into main Jun 4, 2025
56 checks passed
@jhamon jhamon deleted the jhamon/fix-docs-build branch June 4, 2025 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants