| Home | Posts RSS | Comments RSS | Edit


Custom Search
CO.CC:Free Domain

Jumat, 30 Juli 2010

(sedikit) patch untuk lusca cache

Sebelumnya terima kasih untuk mereka-mereka yang telah susah payah membuat agar Lusca cache semakin sempurna dengan membuat patch-patch.
sekarang ijinkan saya untuk mencopas karya mereka. Adapun patch-patch ini memiliki kegunaan antara lain:

1. Ignore-Must-Revalidate patch.
2. Improve %nn parser patch.
3. Memoryleak on maformed requests patch.
4. Async request doesn't use store_url when available patch.
5. HTTP responses with no Date patch.
6. Squid crashes on assert patch.
7. Correct If-(None-)Match: * processing patch.
8. -F switch doesn't block requests while COSS store is being rebuilt patch.
9. Removes Cache-Control request headers, don’t let clients by-pass cache if it is primed.
10. Normalize Accept-Encoding Headers for a higher cache hit rate.
11. Clear Accept-Encoding Headers for content that should not be compressed such as image,video and audio.
berikut ini patchnya..

*** lib/rfc1738.c 2009-11-05 11:56:18.000000000 +0700
--- lib/rfc1738.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 204,233 ****
* rfc1738_unescape() - Converts escaped characters (%xy numbers) in
* given the string. %% is a %. %ab is the 8-bit hexadecimal number "ab"
*/
void
! rfc1738_unescape(char *s)
{
! char hexnum[3];
int i, j; /* i is write, j is read */
! unsigned int x;
for (i = j = 0; s[j]; i++, j++) {
s[i] = s[j];
! if (s[i] != '%')
! continue;
! if (s[j + 1] == '%') { /* %% case */
! j++;
! continue;
! }
! if (s[j + 1] && s[j + 2]) {
! if (s[j + 1] == '0' && s[j + 2] == '0') { /* case */
! j += 2;
! continue;
! }
! hexnum[0] = s[j + 1];
! hexnum[1] = s[j + 2];
! hexnum[2] = '\0';
! if (1 == sscanf(hexnum, "%x", &x)) {
! s[i] = (char) (0x0ff & x);
j += 2;
}
}
--- 204,247 ----
* rfc1738_unescape() - Converts escaped characters (%xy numbers) in
* given the string. %% is a %. %ab is the 8-bit hexadecimal number "ab"
*/
+ static inline int
+ fromhex(char ch)
+ {
+ if (ch >= '0' && ch <= '9')
+ return ch - '0';
+ if (ch >= 'a' && ch <= 'f')
+ return ch - 'a' + 10;
+ if (ch >= 'A' && ch <= 'F')
+ return ch - 'A' + 10;
+ return -1;
+ }
+
void
! rfc1738_unescape(char *s_)
{
! /* char hexnum[3]; */
! unsigned char *s = (unsigned char *) s_;
int i, j; /* i is write, j is read */
! /* unsigned int x; */
for (i = j = 0; s[j]; i++, j++) {
s[i] = s[j];
! if (s[j] != '%') {
! /* normal case, nothing more to do */
! } else if (s[j + 1] == '%') { /* %% case */
! j++; /* Skip % */
! } else {
! /* decode */
! int v1, v2, x;
! v1 = fromhex(s[j + 1]);
! if (v2 < 0)
! continue; /* non-hex or \0 */
! v2 = fromhex(s[j + 2]);
! if (v2 < 0)
! continue; /* non-hex or \0 */
! /* fromhex returns -1 on error which brings this out of range (|, not +) */
! x = v1 << 4 | v2;
! if (x > 0 && x <= 255) {
! s[i] = x;
j += 2;
}
}

*** src/cache_cf.c 2010-02-16 18:21:51.000000000 +0700
--- src/cache_cf.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 489,494 ****
--- 489,500 ----
break;
}
for (R = Config.Refresh; R; R = R->next) {
+ if (!R->flags.ignore_must_revalidate)
+ continue;
+ debug(22, 1) ("WARNING: use of 'ignore-must-revalidate' in 'refresh_pattern' violates HTTP\n");
+ break;
+ }
+ for (R = Config.Refresh; R; R = R->next) {
if (R->stale_while_revalidate <= 0)
continue;
debug(22, 1) ("WARNING: use of 'stale-while-revalidate' in 'refresh_pattern' violates HTTP\n");
***************
*** 2261,2266 ****
--- 2267,2274 ----
storeAppendPrintf(entry, " ignore-private");
if (head->flags.ignore_auth)
storeAppendPrintf(entry, " ignore-auth");
+ if (head->flags.ignore_must_revalidate)
+ storeAppendPrintf(entry, " ignore-must-revalidate");
if (head->stale_while_revalidate > 0)
storeAppendPrintf(entry, " stale-while-revalidate=%d", head->stale_while_revalidate);
#endif
***************
*** 2293,2298 ****
--- 2301,2307 ----
int ignore_no_cache = 0;
int ignore_no_store = 0;
int ignore_private = 0;
+ int ignore_must_revalidate = 0;
int ignore_auth = 0;
#endif
int stale_while_revalidate = -1;
***************
*** 2338,2343 ****
--- 2347,2354 ----
ignore_private = 1;
else if (!strcmp(token, "ignore-auth"))
ignore_auth = 1;
+ else if (!strcmp(token, "ignore-must-revalidate"))
+ ignore_must_revalidate = 1;
else if (!strcmp(token, "reload-into-ims")) {
reload_into_ims = 1;
refresh_nocache_hack = 1;
***************
*** 2397,2402 ****
--- 2408,2415 ----
t->flags.ignore_no_store = 1;
if (ignore_private)
t->flags.ignore_private = 1;
+ if (ignore_must_revalidate)
+ t->flags.ignore_must_revalidate = 1;
if (ignore_auth)
t->flags.ignore_auth = 1;
#endif

*** src/cf.data.pre 2010-03-25 21:25:33.000000000 +0700
--- src/cf.data.pre 2010-07-09 08:31:41.000000000 +0700
***************
*** 3125,3130 ****
--- 3125,3131 ----
ignore-reload
ignore-no-cache
ignore-no-store
+ ignore-must-revalidate
ignore-private
ignore-auth
stale-while-revalidate=NN
***************
*** 3164,3169 ****
--- 3165,3175 ----
from a server, only a client, though plenty of servers
send it anyway.

+ ignore-must-revalidate ignores any ``Cache-Control: must-revalidate''
+ headers received from a server. Doing this VIOLATES
+ the HTTP standard. Enabling this feature could make you
+ liable for problems which it causes.
+
ignore-private ignores any ``Cache-control: private''
headers received from a server. Doing this VIOLATES
the HTTP standard. Enabling this feature could make you

*** src/client_side_async_refresh.c 2010-05-20 16:19:09.000000000 +0700
--- src/client_side_async_refresh.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 76,81 ****
--- 76,83 ----
accessLogLog(&al, ch);
aclChecklistFree(ch);
storeClientUnregister(async->sc, async->entry, async);
+ storeUnlockObject(async->entry->mem_obj->old_entry);
+ async->entry->mem_obj->old_entry = NULL;
storeUnlockObject(async->entry);
storeUnlockObject(async->old_entry);
requestUnlink(async->request);
***************
*** 129,134 ****
--- 131,138 ----
async->entry = storeCreateEntry(url,
request->flags,
request->method);
+ if (request->store_url)
+ storeEntrySetStoreUrl(async->entry, request->store_url);
async->entry->mem_obj->old_entry = async->old_entry;
storeLockObject(async->entry->mem_obj->old_entry);
async->sc = storeClientRegister(async->entry, async);

*** src/client_side.c 2010-07-08 14:22:01.000000000 +0700
--- src/client_side.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 975,980 ****
--- 975,1005 ----
return 1;
}

+ /*
+ * Internal helper function for If-(None-)Match logics
+ */
+ static int
+ checkIfMatch(request_t * request, MemObject * mem, http_hdr_type hdr)
+ {
+ String req_etags;
+ const char *rep_etag;
+ int etag_match = 0;
+
+ if (mem->reply->sline.status != HTTP_OK) {
+ debug(33, 4) ("checkIfMatch: Reply code %d != 200\n",
+ mem->reply->sline.status);
+ return -1; /* Can't check */
+ }
+ rep_etag = httpHeaderGetStr(&mem->reply->header, HDR_ETAG);
+ req_etags = httpHeaderGetList(&request->header, hdr);
+ if (rep_etag)
+ etag_match = strListIsMember(&req_etags, rep_etag, ',');
+ if (!etag_match)
+ etag_match = strListIsMember(&req_etags, "*", ',');
+ stringClean(&req_etags);
+ return etag_match;
+ }
+
/*
* clientCacheHit should only be called until the HTTP reply headers
* have been parsed. Normally this should be a single call, but
***************
*** 1098,1142 ****
return;
}
if (httpHeaderHas(&r->header, HDR_IF_MATCH)) {
! const char *rep_etag = httpHeaderGetStr(&e->mem_obj->reply->header, HDR_ETAG);
! int has_etag = 0;
! if (rep_etag) {
! String req_etags = httpHeaderGetList(&http->request->header, HDR_IF_MATCH);
! has_etag = strListIsMember(&req_etags, rep_etag, ',');
! stringClean(&req_etags);
! }
! if (!has_etag) {
/* The entity tags does not match. This cannot be a hit for this object.
* Query the origin to see what should be done.
*/
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
}
if (httpHeaderHas(&r->header, HDR_IF_NONE_MATCH)) {
! String req_etags;
! const char *rep_etag = httpHeaderGetStr(&e->mem_obj->reply->header, HDR_ETAG);
! int has_etag;
! if (mem->reply->sline.status != HTTP_OK) {
! debug(33, 4) ("clientCacheHit: Reply code %d != 200\n",
! mem->reply->sline.status);
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
! if (rep_etag) {
! req_etags = httpHeaderGetList(&http->request->header, HDR_IF_NONE_MATCH);
! has_etag = strListIsMember(&req_etags, rep_etag, ',');
! stringClean(&req_etags);
! if (has_etag) {
! debug(33, 4) ("clientCacheHit: If-None-Match matches\n");
! if (is_modified == -1)
! is_modified = 0;
! } else {
! debug(33, 4) ("clientCacheHit: If-None-Match mismatch\n");
! is_modified = 1;
! }
}
}
if (r->flags.ims && mem->reply->sline.status == HTTP_OK) {
--- 1123,1156 ----
return;
}
if (httpHeaderHas(&r->header, HDR_IF_MATCH)) {
! int etag_match = checkIfMatch(r, mem, HDR_IF_MATCH);
!
! if (etag_match != 1) {
/* The entity tags does not match. This cannot be a hit for this object.
* Query the origin to see what should be done.
*/
+ + debug(33, 4) ("clientCacheHit: If-Match mismatch\n");
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
}
if (httpHeaderHas(&r->header, HDR_IF_NONE_MATCH)) {
! int etag_match = checkIfMatch(r, mem, HDR_IF_NONE_MATCH);
!
! if (etag_match == -1) {
! debug(33, 4) ("clientCacheHit: If-None-Match failure\n");
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
! if (etag_match) {
! debug(33, 4) ("clientCacheHit: If-None-Match matches\n");
! if (is_modified == -1)
! is_modified = 0;
! } else {
! debug(33, 4) ("clientCacheHit: If-None-Match mismatch\n");
! is_modified = 1;
}
}
if (r->flags.ims && mem->reply->sline.status == HTTP_OK) {
***************
*** 1156,1175 ****
* where the redirect is not explicitly as uncachable.
* Deny looping here and do not cache the response.
*/
- #if 0
/*
* XXX strcmp() sucks but the strings are both C strings. Look at String'ifying it
* XXX soon!
*/
! if (mem->reply->sline.status >= 300 && mem->reply->sline.status < 400) {
! if (!strcmp(http->uri, httpHeaderGetStr(&e->mem_obj->reply->header, HDR_LOCATION))) {
! debug(33, 1) ("clientCacheHit: Redirect Loop Detected: %s\n",http->uri);
! http->log_type = LOG_TCP_MISS;
! clientProcessMiss(http);
! return;
! }
! }
! #endif
stale = refreshCheckHTTPStale(e, r);
debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned %d\n", stale);
if (stale == 0) {
--- 1170,1188 ----
* where the redirect is not explicitly as uncachable.
* Deny looping here and do not cache the response.
*/
/*
* XXX strcmp() sucks but the strings are both C strings. Look at String'ifying it
* XXX soon!
*/
! if (mem->reply->sline.status >= 300 && mem->reply->sline.status < 400) {
! if (httpHeaderHas(&e->mem_obj->reply->header, HDR_LOCATION))
! if (!strcmp(http->uri,httpHeaderGetStr(&e->mem_obj->reply->header, HDR_LOCATION))) {
! debug(33, 2) ("clientCacheHit: Redirect Loop Detected: %s\n",http->uri);
! http->log_type = LOG_TCP_MISS;
! clientProcessMiss(http);
! return;
! }
! }
stale = refreshCheckHTTPStale(e, r);
debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned %d\n", stale);
if (stale == 0) {

*** src/client_side_etag.c 2010-02-14 14:34:59.000000000 +0700
--- src/client_side_etag.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 63,69 ****
if (etag && vary) {
char *str;
str = stringDupToC(&request->vary_encoding);
! storeAddVary(url, entry->mem_obj->method, NULL, httpHeaderGetStr(&rep->header, HDR_ETAG), request->vary_hdr, request->vary_headers, str);
safe_free(str);
}
}
--- 63,69 ----
if (etag && vary) {
char *str;
str = stringDupToC(&request->vary_encoding);
! storeAddVary(entry->mem_obj->store_url, entry->mem_obj->url, entry->mem_obj->method, NULL, httpHeaderGetStr(&rep->header, HDR_ETAG), request->vary_hdr, request->vary_headers, str);
safe_free(str);
}
}

*** src/client_side_ims.c 2010-02-19 14:57:27.000000000 +0700
--- src/client_side_ims.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 164,175 ****
/* If the ETag matches the clients If-None-Match, then return
* the servers 304 reply
*/
! if (httpHeaderHas(&new_entry->mem_obj->reply->header, HDR_ETAG) &&
! httpHeaderHas(&request->header, HDR_IF_NONE_MATCH)) {
! const char *etag = httpHeaderGetStr(&new_entry->mem_obj->reply->header, HDR_ETAG);
String etags = httpHeaderGetList(&request->header, HDR_IF_NONE_MATCH);
! int etag_match = strListIsMember(&etags, etag, ',');
stringClean(&etags);
if (etag_match) {
debug(33, 5) ("clientGetsOldEntry: NO, client If-None-Match\n");
return 0;
--- 164,182 ----
/* If the ETag matches the clients If-None-Match, then return
* the servers 304 reply
*/
! if (httpHeaderHas(&request->header, HDR_IF_NONE_MATCH)) {
String etags = httpHeaderGetList(&request->header, HDR_IF_NONE_MATCH);
! int etag_match = 0;
!
! if (httpHeaderHas(&new_entry->mem_obj->reply->header, HDR_ETAG)) {
! const char *etag = httpHeaderGetStr(&new_entry->mem_obj->reply->header, HDR_ETAG);
! etag_match = strListIsMember(&etags, etag, ',');
! }
! if (!etag_match && strListIsMember(&etags, "*", ','))
! etag_match = 1;
!
stringClean(&etags);
+
if (etag_match) {
debug(33, 5) ("clientGetsOldEntry: NO, client If-None-Match\n");
return 0;

*** src/fs/coss/store_io_coss.c 2009-07-27 10:14:38.000000000 +0700
--- src/fs/coss/store_io_coss.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 157,166 ****

/* Check to see if we need to allocate a membuf to start */
if (cs->current_membuf == NULL) {
! if (cs->curstripe < (cs->numstripes - 1))
! newmb = storeCossCreateMemBuf(SD, cs->curstripe + 1, checkf, &coll);
! else
! newmb = storeCossCreateMemBuf(SD, 0, checkf, &coll);

cs->current_membuf = newmb;
if (newmb == NULL) {
--- 157,166 ----

/* Check to see if we need to allocate a membuf to start */
if (cs->current_membuf == NULL) {
! int nextstripe = cs->curstripe + 1;
! if (nextstripe >= cs->numstripes)
! nextstripe = 0;
! newmb = storeCossCreateMemBuf(SD, nextstripe, checkf, &coll);

cs->current_membuf = newmb;
if (newmb == NULL) {
***************
*** 169,215 ****
}
cs->current_offset = cs->current_membuf->diskstart;

! /* Check if we have overflowed the disk .. */
! } else if ((cs->current_offset + allocsize) > ((off_t) SD->max_size << 10)) {
! /*
! * tried to allocate past the end of the disk, so wrap
! * back to the beginning
! */
! coss_stats.disk_overflows++;
! cs->current_membuf->flags.full = 1;
! cs->numfullstripes++;
! cs->current_membuf->diskend = cs->current_offset;
! storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
! /* cs->current_membuf may be invalid at this point */
! cs->current_offset = 0; /* wrap back to beginning */
! debug(79, 2) ("storeCossAllocate: %s: wrap to 0\n", stripePath(SD));
!
! newmb = storeCossCreateMemBuf(SD, 0, checkf, &coll);
! cs->current_membuf = newmb;
! if (newmb == NULL) {
! cs->sizerange_max = SD->max_objsize;
! return -1;
! }
! /* Check if we have overflowed the MemBuf */
! } else if ((cs->current_offset + allocsize) >= cs->current_membuf->diskend) {
/*
* Skip the blank space at the end of the stripe. start over.
*/
coss_stats.stripe_overflows++;
cs->current_membuf->flags.full = 1;
cs->numfullstripes++;
! cs->current_offset = cs->current_membuf->diskend;
storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
/* cs->current_membuf may be invalid at this point */
debug(79, 3) ("storeCossAllocate: %s: New offset - %" PRId64 "\n", stripePath(SD),
(int64_t) cs->current_offset);
! assert(cs->curstripe < (cs->numstripes - 1));
! newmb = storeCossCreateMemBuf(SD, cs->curstripe + 1, checkf, &coll);
cs->current_membuf = newmb;
if (newmb == NULL) {
cs->sizerange_max = SD->max_objsize;
return -1;
}
}
/* If we didn't get a collision, then update the current offset and return it */
if (coll == 0) {
--- 169,211 ----
}
cs->current_offset = cs->current_membuf->diskstart;

! } else if ((cs->current_offset + allocsize) > cs->current_membuf->diskend) {
/*
* Skip the blank space at the end of the stripe. start over.
*/
+ int nextstripe = cs->curstripe + 1;
coss_stats.stripe_overflows++;
cs->current_membuf->flags.full = 1;
cs->numfullstripes++;
! /* Check if we have overflowed the disk .. */
! if (nextstripe >= cs->numstripes) {
! /*
! * tried to allocate past the end of the disk, so wrap
! * back to the beginning
! */
! debug(79, 2) ("storeCossAllocate: %s: wrap to 0\n", stripePath(SD));
! nextstripe = 0; /* wrap back to beginning */
! coss_stats.disk_overflows++;
! #if LOOKS_WRONG
! /* Original disk wrap code also had this, but looks wrong to
! * me as it leaves garbage at the end of the disk. Either we
! * should do it in both cases, or not at all
! */
! cs->current_membuf->diskend = cs->current_offset;
! #endif
! }
storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
/* cs->current_membuf may be invalid at this point */
debug(79, 3) ("storeCossAllocate: %s: New offset - %" PRId64 "\n", stripePath(SD),
(int64_t) cs->current_offset);
! assert(nextstripe <>numstripes);
! newmb = storeCossCreateMemBuf(SD, nextstripe, checkf, &coll);
cs->current_membuf = newmb;
if (newmb == NULL) {
cs->sizerange_max = SD->max_objsize;
return -1;
}
+ cs->current_offset = cs->current_membuf->diskstart;
}
/* If we didn't get a collision, then update the current offset and return it */
if (coll == 0) {

*** src/http.c 2010-04-20 12:01:43.000000000 +0700
--- src/http.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 334,352 ****
*/
if (!refreshIsCachable(httpState->entry) && !REFRESH_OVERRIDE(store_stale))
return 0;
- /* don't cache objects from peers w/o LMT, Date, or Expires */
- /* check that is it enough to check headers @?@ */
- if (rep->date > -1)
- return 1;
- else if (rep->last_modified > -1)
- return 1;
- else if (!httpState->peer)
- return 1;
- /* @?@ (here and 302): invalid expires header compiles to squid_curtime */
- else if (rep->expires > -1)
- return 1;
else
! return 0;
/* NOTREACHED */
break;
/* Responses that only are cacheable if the server says so */
--- 334,341 ----
*/
if (!refreshIsCachable(httpState->entry) && !REFRESH_OVERRIDE(store_stale))
return 0;
else
! return 1;
/* NOTREACHED */
break;
/* Responses that only are cacheable if the server says so */

***************
*** 926,932 ****
*/
if (len > 0 && httpState->chunk_size == 0) {
if (Config.onoff.log_http_violations)
! debug(11, 1) ("httpReadReply: Unexpected reply body data from \"%s %s\"\n",
urlMethodGetConstStr(orig_request->method), storeUrl(entry));
comm_close(fd);
return;
--- 926,933 ----
*/
if (len > 0 && httpState->chunk_size == 0) {
if (Config.onoff.log_http_violations)
! debug(11, Config.onoff.relaxed_header_parser <= 0 || keep_alive ? 1 : 2)
! ("httpReadReply: Unexpected reply body data from \"%s %s\"\n",
urlMethodGetConstStr(orig_request->method), storeUrl(entry));
comm_close(fd);
return;
*** src/refresh.c 2010-04-10 13:40:41.000000000 +0700
--- src/refresh.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 274,281 ****
debug(22, 3) ("\tcheck_time:\t%s\n", mkrfc1123(check_time));
debug(22, 3) ("\tentry->timestamp:\t%s\n", mkrfc1123(entry->timestamp));

! if (EBIT_TEST(entry->flags, ENTRY_REVALIDATE) && staleness > -1) {
! debug(22, 3) ("refreshCheck: YES: Must revalidate stale response\n");
return STALE_MUST_REVALIDATE;
}
/* request-specific checks */
--- 274,285 ----
debug(22, 3) ("\tcheck_time:\t%s\n", mkrfc1123(check_time));
debug(22, 3) ("\tentry->timestamp:\t%s\n", mkrfc1123(entry->timestamp));

! if (EBIT_TEST(entry->flags, ENTRY_REVALIDATE) && staleness > -1
! #if HTTP_VIOLATIONS
! && !R->flags.ignore_must_revalidate
! #endif
! ) {
! debug(22, 3) ("refreshCheck: YES: Must revalidate stale response\n");
return STALE_MUST_REVALIDATE;
}
/* request-specific checks */
***************
*** 334,339 ****
--- 338,353 ----
* At this point the response is stale, unless one of
* the override options kicks in.
*/
+ #if HTTP_VIOLATIONS
+ if (sf.expires && R->flags.override_expire && age <>min) {
+ debug(22, 3) ("refreshCheck: NO: age < min && override-expire\n");
+ return FRESH_OVERRIDE_EXPIRES;
+ }
+ if (sf.lmfactor && R->flags.override_lastmod && age <>min) {
+ debug(22, 3) ("refreshCheck: NO: age < min && override-lastmod\n");
+ return FRESH_OVERRIDE_LASTMOD;
+ }
+ #endif
if (entry->mem_obj) {
int stale_while_revalidate = -1;
if (entry->mem_obj->reply && entry->mem_obj->reply->cache_control && EBIT_TEST(entry->mem_obj->reply->cache_control->mask, CC_STALE_WHILE_REVALIDATE))
***************
*** 357,382 ****
if (delta < 0 && staleness + delta < 0) {
return STALE_WITHIN_DELTA;
}
! if (sf.expires) {
! #if HTTP_VIOLATIONS
! if (R->flags.override_expire && age <>min) {
! debug(22, 3) ("refreshCheck: NO: age < min && override-expire\n");
! return FRESH_OVERRIDE_EXPIRES;
! }
! #endif
return STALE_EXPIRES;
- }
if (sf.max)
return STALE_MAX_RULE;
! if (sf.lmfactor) {
! #if HTTP_VIOLATIONS
! if (R->flags.override_lastmod && age <>min) {
! debug(22, 3) ("refreshCheck: NO: age < min && override-lastmod\n");
! return FRESH_OVERRIDE_LASTMOD;
! }
! #endif
return STALE_LMFACTOR_RULE;
- }
return STALE_DEFAULT;
}

--- 371,382 ----
if (delta < 0 && staleness + delta < 0) {
return STALE_WITHIN_DELTA;
}
! if (sf.expires)
return STALE_EXPIRES;
if (sf.max)
return STALE_MAX_RULE;
! if (sf.lmfactor)
return STALE_LMFACTOR_RULE;
return STALE_DEFAULT;
}

*** src/store.c 2010-03-06 09:36:15.000000000 +0700
--- src/store.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 553,559 ****
stringClean(&varyhdr);
#endif
str = stringDupToC(&vary);
! storeAddVary(mem->url, mem->method, newkey, httpHeaderGetStr(&mem->reply->header, HDR_ETAG), str, mem->vary_headers, mem->vary_encoding);
safe_free(str);
stringClean(&vary);
}
--- 553,559 ----
stringClean(&varyhdr);
#endif
str = stringDupToC(&vary);
! storeAddVary(mem->store_url, mem->url, mem->method, newkey, httpHeaderGetStr(&mem->reply->header, HDR_ETAG), str, mem->vary_headers, mem->vary_encoding);
safe_free(str);
stringClean(&vary);
}

*** src/store_client.c 2009-04-21 15:37:50.000000000 +0700
--- src/store_client.c 2010-11-24 09:51:46.000000000 +0700
***************
*** 894,905 ****
debug(20, 3) ("CheckQuickAbort2: YES !mem->request->flags.cachable\n");
return 1;
}
if (EBIT_TEST(entry->flags, KEY_PRIVATE)) {
debug(20, 3) ("CheckQuickAbort2: YES KEY_PRIVATE\n");
return 1;
}
- expectlen = mem->reply->content_length + mem->reply->hdr_sz;
- curlen = mem->inmem_hi;
minlen = Config.quickAbort.min << 10;
if (minlen < 0) {
debug(20, 3) ("CheckQuickAbort2: NO disabled\n");
--- 894,909 ----
debug(20, 3) ("CheckQuickAbort2: YES !mem->request->flags.cachable\n");
return 1;
}
+ expectlen = httpReplyBodySize(mem->method, mem->reply) + mem->reply->hdr_sz;
+ curlen = mem->inmem_hi;
+ if (expectlen == curlen) {
+ debug(20, 3) ("CheckQuickAbort2: NO already finished\n");
+ return 0;
+ }
if (EBIT_TEST(entry->flags, KEY_PRIVATE)) {
debug(20, 3) ("CheckQuickAbort2: YES KEY_PRIVATE\n");
return 1;
}
minlen = Config.quickAbort.min << 10;
if (minlen < 0) {
debug(20, 3) ("CheckQuickAbort2: NO disabled\n");

*** src/store_vary.c 2010-10-19 09:22:25.000000000 +0700
--- src/store_vary.c 2010-11-24 12:09:37.000000000 +0700
***************
*** 6,11 ****
--- 6,12 ----
StoreEntry *oe;
StoreEntry *e;
store_client *sc;
+ char *store_url;
char *url;
char *key;
char *vary_headers;
***************
*** 57,62 ****
--- 58,64 ----
storeUnlockObject(state->oe);
state->oe = NULL;
}
+ safe_free(state->store_url);
safe_free(state->url);
safe_free(state->key);
safe_free(state->vary_headers);
***************
*** 312,318 ****
* At leas one of key or etag must be specified, preferably both.
*/
void
! storeAddVary(const char *url, method_t * method, const cache_key * key, const char *etag, const char *vary, const char *vary_headers, const char *accept_encoding)
{
AddVaryState *state;
request_flags flags = null_request_flags;
--- 314,320 ----
* At leas one of key or etag must be specified, preferably both.
*/
void
! storeAddVary(const char *store_url, const char *url, method_t * method, const cache_key * key, const char *etag, const char *vary, const char *vary_headers, const char *accept_encoding)
{
AddVaryState *state;
request_flags flags = null_request_flags;
***************
*** 326,332 ****
state->accept_encoding = xstrdup(accept_encoding);
if (etag)
state->etag = xstrdup(etag);
! state->oe = storeGetPublic(url, method);
debug(11, 2) ("storeAddVary: %s (%s) %s %s\n",
state->url, state->key, state->vary_headers, state->etag);
if (state->oe)
--- 328,334 ----
state->accept_encoding = xstrdup(accept_encoding);
if (etag)
state->etag = xstrdup(etag);
! state->oe = storeGetPublic(store_url ? store_url : url, method);
debug(11, 2) ("storeAddVary: %s (%s) %s %s\n",
state->url, state->key, state->vary_headers, state->etag);
if (state->oe)

*** src/store_vary.h 2010-02-14 14:34:59.000000000 +0700
--- src/store_vary.h 2010-07-09 08:31:42.000000000 +0700
***************
*** 4,10 ****
extern void storeLocateVaryDone(VaryData * data);
extern void storeLocateVary(StoreEntry * e, int offset, const char *vary_data,
String accept_encoding, STLVCB * callback, void *cbdata);
! extern void storeAddVary(const char *url, method_t * method, const cache_key * key,
const char *etag, const char *vary, const char *vary_headers,
const char *accept_encoding);

--- 4,10 ----
extern void storeLocateVaryDone(VaryData * data);
extern void storeLocateVary(StoreEntry * e, int offset, const char *vary_data,
String accept_encoding, STLVCB * callback, void *cbdata);
! extern void storeAddVary(const char *store_url, const char *url, method_t * method, const cache_key * key,
const char *etag, const char *vary, const char *vary_headers,
const char *accept_encoding);

*** src/structs.h 2010-04-21 21:10:06.000000000 +0700
--- src/structs.h 2010-07-09 08:31:42.000000000 +0700
***************
*** 1706,1712 ****
unsigned int reload_into_ims:1;
unsigned int ignore_reload:1;
unsigned int ignore_no_cache:1;
! unsigned int ignore_no_store:1;
unsigned int ignore_private:1;
unsigned int ignore_auth:1;
#endif
--- 1706,1713 ----
unsigned int reload_into_ims:1;
unsigned int ignore_reload:1;
unsigned int ignore_no_cache:1;
! unsigned int ignore_no_store:1;
! unsigned int ignore_must_revalidate:1;
unsigned int ignore_private:1;
unsigned int ignore_auth:1;
#endif

*** src/main.c 2010-04-21 21:10:06.000000000 +0700
--- src/main.c 2010-07-12 09:35:25.444745821 +0700
***************
*** 581,587 ****

_db_init(Config.debugOptions);
_db_init_log(Config.Log.log);
! fd_open(fileno(debug_log), FD_LOG, Config.Log.log);
#if MEM_GEN_TRACE
log_trace_init("/tmp/squid.alloc");
#endif
--- 581,588 ----

_db_init(Config.debugOptions);
_db_init_log(Config.Log.log);
! if (debug_log != stderr)
! fd_open(fileno(debug_log), FD_LOG, Config.Log.log);
#if MEM_GEN_TRACE
log_trace_init("/tmp/squid.alloc");
#endif

*** src/main.c 2010-04-21 21:10:06.000000000 +0700
--- src/main.c 2010-07-15 09:08:53.385643957 +0700
***************
*** 682,688 ****
#if USE_WCCPv2
wccp2Init();
#endif
! serverConnectionsOpen();
neighbors_init();
if (Config.chroot_dir)
no_suid();
--- 682,689 ----
#if USE_WCCPv2
wccp2Init();
#endif
! if (!opt_foreground_rebuild)
! serverConnectionsOpen();
neighbors_init();
if (Config.chroot_dir)
no_suid();
***************
*** 911,916 ****
--- 912,922 ----
#endif
serverConnectionsClose();
eventAdd("SquidShutdown", SquidShutdown, NULL, (double) (wait + 1), 1);
+ } else if (opt_foreground_rebuild && !store_dirs_rebuilding) {
+ opt_foreground_rebuild = 0;
+ enter_suid();
+ serverConnectionsOpen();
+ leave_suid();
}
/* Set a maximum loop delay; it'll be lowered elsewhere as appropriate */
loop_delay = 60000;

*** src/client_side_request_parse.c 2010-04-05 14:40:47.000000000 +0700
--- src/client_side_request_parse.c 2010-07-27 11:01:51.520075515 +0700
***************
*** 516,521 ****
--- 516,569 ----
ret = -1;
goto finish;
}
+
+ /*
+ * Normalize Request Cache-Control / If-Modified-Since Headers
+ * Don't let client by-pass the cache if there is cached content.
+ */
+ if(httpHeaderHas(&request->header,HDR_CACHE_CONTROL)) {
+ httpHeaderDelByName(&request->header,"cache-control");
+ }
+
+ /*
+ * Un-comment this if you want Squid to always respond with the request
+ * instead of returning back with a 304 if the cache has not changed.
+ */
+
+ if(httpHeaderHas(&request->header,HDR_IF_MODIFIED_SINCE)) {
+ httpHeaderDelByName(&request->header,"if-modified-since");
+ }
+
+ /*
+ * Normalize Accept-Encoding Headers sent from client
+ */
+ if(httpHeaderHas(&request->header,HDR_ACCEPT_ENCODING)) {
+ String val = httpHeaderGetByName(&request->header,"accept-encoding");
+ if(val.buf) {
+ if(strstr(val.buf,"gzip") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ httpHeaderPutStr(&request->header,HDR_ACCEPT_ENCODING,"gzip");
+ } else if(strstr(val.buf,"deflate") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ httpHeaderPutStr(&request->header,HDR_ACCEPT_ENCODING,"deflate");
+ } else {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ }
+ }
+ stringClean(&val);
+ }
+
+ /*
+ * Normalize Accept-Encoding Headers for video/image content
+ */
+ char *mime_type = mimeGetContentType(http->uri);
+ if(mime_type) {
+ if(strstr(mime_type,"image") != NULL || strstr(mime_type,"video") != NULL || strstr(mime_type,"audio") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ }
+ }
+
+
/*
* If we read past the end of this request, move the remaining
* data to the beginning




atau bisa donlot disini

Read More......

Selasa, 27 Juli 2010

yuk mainan lusca cache di freebsd....

Setelah sekian lama "bermain-main" dengan squid 2.7 STABLE7, iseng iseng mencoba lusca cache besutan Adrian Chadd. Lusca cache sebenarnya masih kerabatan sama squid 2.7 soalnya lusca dikembangkan dari sana. Perbedaannya, lusca salah satunya mencoba menitikberatkan peningkatan performa dalam peyimpanan cache ke hardisk dan beberapa script dicoba ditata ulang untuk meningkatkan performanya. Salah satu hasil signifikan adalah Modul COSS, dimana COSS lusca proses rebuildingnya lebih cepet dibanding COSS di squid aslinya.

Ok, back to topik again..
karena lusca ini nantinya akan berjalan di freebsd (saya pake freebsd 8.0) maka sudah tentu harus sukses dulu install freebsdnya :D
Untuk hanya menjalankan lusca cache difreebsd, paket yang dibutuhkan hanya perl. Anda bisa menginstallnya via port. Agar lebih "menyenangkan" , segala proses install menggunakan putty dan winscp jadi pengerjaannya via remote dari komputer/laptop basis win***s.

1. install perl via port

#cd /usr/ports/lang/perl5.10
#make install clean


tunggu beberapa saat, untuk mengetest apakah perl sudah terinstall ketikan saja diterminal perl -v

2. Download source lusca cache.

Source lusca cache versi terakhir bisa di donlot di code.google.com/p/lusca-cache/. Setelah di donlot silahkan di trasnfer ke mesin freebsd memakai winscp. Anda bisa menaruhnya di /usr/local/src
donlot juga patch ini lusca-patch nanti diesktrak dan taruh juga di /usr/local/src

3. Kompilasi lusca cache.

Setelah di transfer ke mesin freebsd via winscp dan diletakan di /usr/local/src selanjutnya kita unpack source dan lakukan patch :

#cd /usr/local/src
#tar -xvf LUSCA_HEAD-rxxxx.tar.gz
#cd LUSCA_HEAD-rxxxx
#patch -p0 < ../lusca-r14723-sum14rdi.patch #./configure --bindir=/usr/local/bin --sbindir=/usr/local/sbin --sysconfdir=/usr/local/etc/squid --datadir=/usr/local/etc/squid --libexecdir=/usr/local/libexec/squid --localstatedir=/var/log/squid --enable-removal-policies="heap" --enable-auth="basic ntlm digest" --enable-digest-auth-helpers=password --with-pthreads --enable-async-io=24 --with-aufs-threads=24 --enable-storeio="aufs coss" --disable-ident-lookups --enable-delay-pools --enable-snmp --enable-cache-digests --disable-wccp --enable-useragent-log --enable-http-violations --enable-arp-acl --enable-pf-transparent --disable-follow-x-forwarded-for --with-large-files --enable-large-cache-files --enable-default-err-language=English #make && make install

4. Running for first time

sebelum squid dijalankan pastikan telh dibuat user dan group squid di freebsd, kemudian squid.conf yang ada di /usr/local/etc/squid diconfigure sesuai kebutuhannya. Anda bisa mencontoh squid.conf saya dan pastikan anda untuk menyesuaikan dengan kondisi jaringan anda.
Jangan lupa untuk mengubah kepemilikan directory untuk cache kepada squid. Setelah itu baru :

# squid -z
# squid -DF

agar squid dapat jalan otomatis ketika restart, maka taruh squid -DF di file /etc/rc.local, jika tidak ada maka silahkan membuatnya.

SELESAI






Read More......

Install FreeBSD ke hardisk (bukan tutorial)

Setelah sekian lama gak tulis-tulis (akibat lupa username dan password ke blog ini) sekarang mencoba menuangkan kembali pengalaman saya. Ini adalah yang pertama pasca ketemunya username dan passwordnya :D

Sebenarnya sudah banyak yang mengulas bagaimana cara menginstall freebsd ke hardisk. Anda bisa "sowan" ke simbah google, salah dua nya yang menurut saya simple ada disini

1. www.freebsd.org
2. freebsd.or.id

secara garis besar menginstall freebsd ke hardisk meliputi beberapa bagian besar yaitu:
1. pemilihan setup
2. pemilihan partisi.
3. pemilihan file system & swap (labeling)
4. pemilihan source
5. memulai proses instalasi

 

Read More......
 
from my XP © Template Design by Herro