Buffers are only added and not removed

[patrickeibl]

source._get_buffers calls opts.get_bufnrs() to get the buffers for indexing. It checks if each buffer is in buffers and adds it if it's not there. It fails to remove buffers that were not returned from opts.get_bufnrs().

I was experiencing the performance issues described in #40 and this was the cause. The number of indexed buffers only grows, regardless of what get_bufnrs() returns.

[patrickeibl]

I'm testing a change to call close() on any buffers in self.buffers that are not returned by opts.get_bufnrs() and will submit a PR if that works.