You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.markdown
+31-34Lines changed: 31 additions & 34 deletions
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,3 @@
1
-
<!---
2
-
Don't edit this file manually! Instead you should generate it by using:
3
-
wiki2markdown.pl doc/HttpSRCacheModule.wiki
4
-
-->
5
-
6
1
Name
7
2
====
8
3
@@ -165,7 +160,7 @@ Description
165
160
166
161
This module provides a transparent caching layer for arbitrary nginx locations (like those use an upstream or even serve static disk files). The caching behavior is mostly compatible with [RFC 2616](http://www.ietf.org/rfc/rfc2616.txt).
167
162
168
-
Usually, [memc-nginx-module](http://github.com/openresty/memc-nginx-module) is used together with this module to provide a concrete caching storage backend. But technically, any modules that provide a REST interface can be used as the fetching and storage subrequests used by this module.
163
+
Usually, [memc-nginx-module](https://github.com/openresty/memc-nginx-module) is used together with this module to provide a concrete caching storage backend. But technically, any modules that provide a REST interface can be used as the fetching and storage subrequests used by this module.
169
164
170
165
For main requests, the [srcache_fetch](#srcache_fetch) directive works at the end of the access phase, so the [standard access module](http://nginx.org/en/docs/http/ngx_http_access_module.html)'s [allow](http://nginx.org/en/docs/http/ngx_http_access_module.html#allow) and [deny](http://nginx.org/en/docs/http/ngx_http_access_module.html#deny) direcives run *before* ours, which is usually the desired behavior for security reasons.
171
166
@@ -180,7 +175,7 @@ Subrequest caching
180
175
181
176
For *subrequests*, we explicitly **disallow** the use of this module because it's too difficult to get right. There used to be an implementation but it was buggy and I finally gave up fixing it and abandoned it.
182
177
183
-
However, if you're using [lua-nginx-module](http://github.com/openresty/lua-nginx-module), it's easy to do subrequest caching in Lua all by yourself. That is, first issue a subrequest to an [memc-nginx-module](http://github.com/openresty/memc-nginx-module) location to do an explicit cache lookup, if cache hit, just use the cached data returned; otherwise, fall back to the true backend, and finally do a cache insertion to feed the data into the cache.
178
+
However, if you're using [lua-nginx-module](https://github.com/openresty/lua-nginx-module), it's easy to do subrequest caching in Lua all by yourself. That is, first issue a subrequest to an [memc-nginx-module](https://github.com/openresty/memc-nginx-module) location to do an explicit cache lookup, if cache hit, just use the cached data returned; otherwise, fall back to the true backend, and finally do a cache insertion to feed the data into the cache.
184
179
185
180
Using this module for main request caching and Lua for subrequest caching is the approach that we're taking in our business. This hybrid solution works great in production.
186
181
@@ -235,16 +230,16 @@ Here is a simple example demonstrating a distributed memcached caching mechanism
235
230
```
236
231
Here's what is going on in the sample above:
237
232
1. We first define three upstreams, `moon`, `earth`, and `sun`. These are our three memcached servers.
238
-
1. And then we group them together as an upstream list entity named `universe` with the `upstream_list` directive provided by [set-misc-nginx-module](http://github.com/openresty/set-misc-nginx-module).
233
+
1. And then we group them together as an upstream list entity named `universe` with the `upstream_list` directive provided by [set-misc-nginx-module](https://github.com/openresty/set-misc-nginx-module).
239
234
1. After that, we define an internal location named `/memc` for talking to the memcached cluster.
240
-
1. In this `/memc` location, we first set the `$memc_key` variable with the query string (`$args`), and then use the [set_hashed_upstream](http://github.com/openresty/set-misc-nginx-module#set_hashed_upstream) directive to hash our [$memc_key](http://github.com/openresty/memc-nginx-module#memc_key) over the upsteam list `universe`, so as to obtain a concrete upstream name to be assigned to the variable `$backend`.
241
-
1. We pass this `$backend` variable into the [memc_pass](http://github.com/openresty/memc-nginx-module#memc_pass) directive. The `$backend` variable can hold a value among `moon`, `earth`, and `sun`.
242
-
1. Also, we define the memcached caching expiration time to be 3600 seconds (i.e., an hour) by overriding the [$memc_exptime](http://github.com/openresty/memc-nginx-module#memc_exptime) variable.
235
+
1. In this `/memc` location, we first set the `$memc_key` variable with the query string (`$args`), and then use the [set_hashed_upstream](https://github.com/openresty/set-misc-nginx-module#set_hashed_upstream) directive to hash our [$memc_key](https://github.com/openresty/memc-nginx-module#memc_key) over the upsteam list `universe`, so as to obtain a concrete upstream name to be assigned to the variable `$backend`.
236
+
1. We pass this `$backend` variable into the [memc_pass](https://github.com/openresty/memc-nginx-module#memc_pass) directive. The `$backend` variable can hold a value among `moon`, `earth`, and `sun`.
237
+
1. Also, we define the memcached caching expiration time to be 3600 seconds (i.e., an hour) by overriding the [$memc_exptime](https://github.com/openresty/memc-nginx-module#memc_exptime) variable.
243
238
1. In our main public location `/`, we configure the `$uri` variable as our cache key, and then configure [srcache_fetch](#srcache_fetch) for cache lookups and [srcache_store](#srcache_store) for cache updates. We're using two subrequests to our `/memc` location defined earlier in these two directives.
244
239
245
-
One can use [lua-nginx-module](http://github.com/openresty/lua-nginx-module)'s [set_by_lua](http://github.com/openresty/lua-nginx-module#set_by_lua) or [rewrite_by_lua](http://github.com/openresty/lua-nginx-module#rewrite_by_lua) directives to inject custom Lua code to compute the `$backend` and/or `$key` variables in the sample above.
240
+
One can use [lua-nginx-module](https://github.com/openresty/lua-nginx-module)'s [set_by_lua](https://github.com/openresty/lua-nginx-module#set_by_lua) or [rewrite_by_lua](https://github.com/openresty/lua-nginx-module#rewrite_by_lua) directives to inject custom Lua code to compute the `$backend` and/or `$key` variables in the sample above.
246
241
247
-
One thing that should be taken care of is that memcached does have restriction on key lengths, i.e., 250 bytes, so for keys that may be very long, one could use the [set_md5](http://github.com/openresty/set-misc-nginx-module#set_md5) directive or its friends to pre-hash the key to a fixed-length digest before assigning it to `$memc_key` in the `/memc` location or the like.
242
+
One thing that should be taken care of is that memcached does have restriction on key lengths, i.e., 250 bytes, so for keys that may be very long, one could use the [set_md5](https://github.com/openresty/set-misc-nginx-module#set_md5) directive or its friends to pre-hash the key to a fixed-length digest before assigning it to `$memc_key` in the `/memc` location or the like.
248
243
249
244
Further, one can utilize the [srcache_fetch_skip](#srcache_fetch_skip) and [srcache_store_skip](#srcache_store_skip) directives to control what to cache and what not on a per-request basis, and Lua can also be used here in a similar way. So the possibility is really unlimited.
250
245
@@ -304,11 +299,11 @@ Here is a working example by using Redis:
304
299
}
305
300
```
306
301
307
-
This example makes use of the [$echo_request_body](http://github.com/openresty/echo-nginx-module#echo_request_body) variable provided by [echo-nginx-module](http://github.com/openresty/echo-nginx-module). Note that you need the latest version of [echo-nginx-module](http://github.com/openresty/echo-nginx-module), `v0.38rc2` because earlier versions may not work reliably.
302
+
This example makes use of the [$echo_request_body](https://github.com/openresty/echo-nginx-module#echo_request_body) variable provided by [echo-nginx-module](https://github.com/openresty/echo-nginx-module). Note that you need the latest version of [echo-nginx-module](https://github.com/openresty/echo-nginx-module), `v0.38rc2` because earlier versions may not work reliably.
308
303
309
-
Also, you need both [HttpRedisModule](http://wiki.nginx.org/HttpRedisModule) and [redis2-nginx-module](http://github.com/openresty/redis2-nginx-module). The former is used in the [srcache_fetch](#srcache_fetch) subrequest and the latter is used in the [srcache_store](#srcache_store) subrequest.
304
+
Also, you need both [HttpRedisModule](http://wiki.nginx.org/HttpRedisModule) and [redis2-nginx-module](https://github.com/openresty/redis2-nginx-module). The former is used in the [srcache_fetch](#srcache_fetch) subrequest and the latter is used in the [srcache_store](#srcache_store) subrequest.
310
305
311
-
The Nginx core also has a bug that could prevent [redis2-nginx-module](http://github.com/openresty/redis2-nginx-module)'s pipelining support from working properly in certain extreme conditions. And the following patch fixes this:
306
+
The Nginx core also has a bug that could prevent [redis2-nginx-module](https://github.com/openresty/redis2-nginx-module)'s pipelining support from working properly in certain extreme conditions. And the following patch fixes this:
we want to remove the `SID` and `UID` arguments from it. It is easy to achieve if you use [lua-nginx-module](http://github.com/openresty/lua-nginx-module) at the same time:
323
+
we want to remove the `SID` and `UID` arguments from it. It is easy to achieve if you use [lua-nginx-module](https://github.com/openresty/lua-nginx-module) at the same time:
329
324
330
325
```nginx
331
326
@@ -341,9 +336,9 @@ we want to remove the `SID` and `UID` arguments from it. It is easy to achieve i
341
336
}
342
337
```
343
338
344
-
Here we use the [echo](http://github.com/openresty/echo-nginx-module#echo) directive from [echo-nginx-module](http://github.com/openresty/echo-nginx-module) to dump out
339
+
Here we use the [echo](https://github.com/openresty/echo-nginx-module#echo) directive from [echo-nginx-module](https://github.com/openresty/echo-nginx-module) to dump out
345
340
the final value of [$args](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_args) in the end. You can replace it with your
346
-
[srcache-nginx-module](http://github.com/openresty/srcache-nginx-module) configurations and upstream configurations instead for
341
+
[srcache-nginx-module](https://github.com/openresty/srcache-nginx-module) configurations and upstream configurations instead for
347
342
your case. Let's test this /t interface with curl:
@@ -436,7 +431,7 @@ For example, to skip caching requests which have a cookie named `foo` with the v
436
431
# proxy_pass/fastcgi_pass/content_by_lua/...
437
432
}
438
433
```
439
-
where [lua-nginx-module](http://github.com/openresty/lua-nginx-module) is used to calculate the value of the `$skip` variable at the (earlier) rewrite phase. Similarly, the `$key` variable can be computed by Lua using the [set_by_lua](http://github.com/openresty/lua-nginx-module#set_by_lua) or [rewrite_by_lua](http://github.com/openresty/lua-nginx-module#rewrite_by_lua) directive too.
434
+
where [lua-nginx-module](https://github.com/openresty/lua-nginx-module) is used to calculate the value of the `$skip` variable at the (earlier) rewrite phase. Similarly, the `$key` variable can be computed by Lua using the [set_by_lua](https://github.com/openresty/lua-nginx-module#set_by_lua) or [rewrite_by_lua](https://github.com/openresty/lua-nginx-module#rewrite_by_lua) directive too.
440
435
441
436
The standard [map](http://nginx.org/en/docs/http/ngx_http_map_module.html#map) directive can also be used to compute the value of the `$skip` variable used in the sample above:
442
437
@@ -944,7 +939,7 @@ This variable was first introduced in the `v0.14` release.
944
939
Known Issues
945
940
============
946
941
* On certain systems, enabling aio and/or sendfile may stop [srcache_store](#srcache_store) from working. You can disable them in the locations configured by [srcache_store](#srcache_store).
947
-
* The [srcache_store](#srcache_store) directive can not be used to capture the responses generated by [echo-nginx-module](http://github.com/openresty/echo-nginx-module)'s subrequest directivees like [echo_subrequest_async](http://github.com/openresty/echo-nginx-module#echo_subrequest_async) and [echo_location](http://github.com/openresty/echo-nginx-module#echo_location). You are recommended to use HttpLuaModule to initiate and capture subrequests, which should work with [srcache_store](#srcache_store).
942
+
* The [srcache_store](#srcache_store) directive can not be used to capture the responses generated by [echo-nginx-module](https://github.com/openresty/echo-nginx-module)'s subrequest directivees like [echo_subrequest_async](https://github.com/openresty/echo-nginx-module#echo_subrequest_async) and [echo_location](https://github.com/openresty/echo-nginx-module#echo_location). You are recommended to use HttpLuaModule to initiate and capture subrequests, which should work with [srcache_store](#srcache_store).
948
943
949
944
[Back to TOC](#table-of-contents)
950
945
@@ -955,7 +950,7 @@ Caveats
955
950
956
951
proxy_set_header Accept-Encoding "";
957
952
```
958
-
* Do *not* use [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [if](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) directive in the same location as this module's, because "[if](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) is evil". Instead, use [ngx_http_map_module](http://nginx.org/en/docs/http/ngx_http_map_module.html) or [lua-nginx-module](http://github.com/openresty/lua-nginx-module) combined with this module's [srcache_store_skip](#srcache_store_skip) and/or [srcache_fetch_skip](#srcache_fetch_skip) directives. For example:
953
+
* Do *not* use [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html)'s [if](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) directive in the same location as this module's, because "[if](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) is evil". Instead, use [ngx_http_map_module](http://nginx.org/en/docs/http/ngx_http_map_module.html) or [lua-nginx-module](https://github.com/openresty/lua-nginx-module) combined with this module's [srcache_store_skip](#srcache_store_skip) and/or [srcache_fetch_skip](#srcache_fetch_skip) directives. For example:
959
954
```nginx
960
955
961
956
map $request_method $skip_fetch {
@@ -1002,15 +997,15 @@ It is recommended to install this module as well as the Nginx core and many othe
1002
997
1003
998
Alternatively, you can build Nginx with this module all by yourself:
1004
999
1005
-
* Grab the nginx source code from [nginx.org](http://nginx.org), for example, the version 1.7.10 (see [Nginx Compatibility](#compatibility)),
1000
+
* Grab the nginx source code from [nginx.org](http://nginx.org), for example, the version 1.9.3 (see [Nginx Compatibility](#compatibility)),
1006
1001
* and then apply the patch to your nginx source tree that fixes an important bug in the mainline Nginx core: <https://raw.github.com/openresty/ngx_openresty/master/patches/nginx-1.4.3-upstream_truncation.patch> (you do NOT need this patch if you are using nginx 1.5.3 and later versions.)
1007
-
* after that, download the latest version of the release tarball of this module from srcache-nginx-module [file list](http://github.com/openresty/srcache-nginx-module/tags),
1002
+
* after that, download the latest version of the release tarball of this module from srcache-nginx-module [file list](https://github.com/openresty/srcache-nginx-module/tags),
1008
1003
* and finally build the Nginx source with this module
# Here we assume you would install you nginx under /opt/nginx/.
1016
1011
./configure --prefix=/opt/nginx \
@@ -1027,6 +1022,8 @@ Compatibility
1027
1022
1028
1023
The following versions of Nginx should work with this module:
1029
1024
1025
+
* 1.9.x (last tested: 1.9.3)
1026
+
* 1.8.x
1030
1027
* 1.7.x (last tested: 1.7.10)
1031
1028
* 1.5.x (last tested: 1.5.12)
1032
1029
* 1.4.x (last tested: 1.4.4)
@@ -1067,20 +1064,20 @@ Bugs and Patches
1067
1064
1068
1065
Please submit bug reports, wishlists, or patches by
1069
1066
1070
-
1. creating a ticket on the [GitHub Issue Tracker](http://github.com/openresty/srcache-nginx-module/issues),
1067
+
1. creating a ticket on the [GitHub Issue Tracker](https://github.com/openresty/srcache-nginx-module/issues),
1071
1068
1. or posting to the [OpenResty community](#community).
1072
1069
1073
1070
[Back to TOC](#table-of-contents)
1074
1071
1075
1072
Source Repository
1076
1073
=================
1077
-
Available on github at [openresty/srcache-nginx-module](http://github.com/openresty/srcache-nginx-module).
1074
+
Available on github at [openresty/srcache-nginx-module](https://github.com/openresty/srcache-nginx-module).
1078
1075
1079
1076
[Back to TOC](#table-of-contents)
1080
1077
1081
1078
Test Suite
1082
1079
==========
1083
-
This module comes with a Perl-driven test suite. The [test cases](http://github.com/openresty/srcache-nginx-module/tree/master/test/t) are [declarative](http://github.com/openresty/srcache-nginx-module/blob/master/test/t/main-req.t) too. Thanks to the [Test::Nginx](http://search.cpan.org/perldoc?Test::Base) module in the Perl world.
1080
+
This module comes with a Perl-driven test suite. The [test cases](https://github.com/openresty/srcache-nginx-module/tree/master/test/t) are [declarative](https://github.com/openresty/srcache-nginx-module/blob/master/test/t/main-req.t) too. Thanks to the [Test::Nginx](http://search.cpan.org/perldoc?Test::Base) module in the Perl world.
1084
1081
1085
1082
To run it on your side:
1086
1083
```bash
@@ -1091,7 +1088,7 @@ You need to terminate any Nginx processes before running the test suite if you h
1091
1088
1092
1089
Because a single nginx server (by default, `localhost:1984`) is used across all the test scripts (`.t` files), it's meaningless to run the test suite in parallel by specifying `-jN` when invoking the `prove` utility.
1093
1090
1094
-
Some parts of the test suite requires modules [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html), [echo-nginx-module](http://github.com/openresty/echo-nginx-module), [rds-json-nginx-module](http://github.com/openresty/rds-json-nginx-module), and [drizzle-nginx-module](http://github.com/openresty/drizzle-nginx-module) to be enabled as well when building Nginx.
1091
+
Some parts of the test suite requires modules [ngx_http_rewrite_module](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html), [echo-nginx-module](https://github.com/openresty/echo-nginx-module), [rds-json-nginx-module](https://github.com/openresty/rds-json-nginx-module), and [drizzle-nginx-module](https://github.com/openresty/drizzle-nginx-module) to be enabled as well when building Nginx.
1095
1092
1096
1093
[Back to TOC](#table-of-contents)
1097
1094
@@ -1132,8 +1129,8 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
0 commit comments