| Home | Posts RSS | Comments RSS | Edit


Custom Search
CO.CC:Free Domain

Jumat, 30 Juli 2010

(sedikit) patch untuk lusca cache

Sebelumnya terima kasih untuk mereka-mereka yang telah susah payah membuat agar Lusca cache semakin sempurna dengan membuat patch-patch.
sekarang ijinkan saya untuk mencopas karya mereka. Adapun patch-patch ini memiliki kegunaan antara lain:

1. Ignore-Must-Revalidate patch.
2. Improve %nn parser patch.
3. Memoryleak on maformed requests patch.
4. Async request doesn't use store_url when available patch.
5. HTTP responses with no Date patch.
6. Squid crashes on assert patch.
7. Correct If-(None-)Match: * processing patch.
8. -F switch doesn't block requests while COSS store is being rebuilt patch.
9. Removes Cache-Control request headers, don’t let clients by-pass cache if it is primed.
10. Normalize Accept-Encoding Headers for a higher cache hit rate.
11. Clear Accept-Encoding Headers for content that should not be compressed such as image,video and audio.
berikut ini patchnya..

*** lib/rfc1738.c 2009-11-05 11:56:18.000000000 +0700
--- lib/rfc1738.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 204,233 ****
* rfc1738_unescape() - Converts escaped characters (%xy numbers) in
* given the string. %% is a %. %ab is the 8-bit hexadecimal number "ab"
*/
void
! rfc1738_unescape(char *s)
{
! char hexnum[3];
int i, j; /* i is write, j is read */
! unsigned int x;
for (i = j = 0; s[j]; i++, j++) {
s[i] = s[j];
! if (s[i] != '%')
! continue;
! if (s[j + 1] == '%') { /* %% case */
! j++;
! continue;
! }
! if (s[j + 1] && s[j + 2]) {
! if (s[j + 1] == '0' && s[j + 2] == '0') { /* case */
! j += 2;
! continue;
! }
! hexnum[0] = s[j + 1];
! hexnum[1] = s[j + 2];
! hexnum[2] = '\0';
! if (1 == sscanf(hexnum, "%x", &x)) {
! s[i] = (char) (0x0ff & x);
j += 2;
}
}
--- 204,247 ----
* rfc1738_unescape() - Converts escaped characters (%xy numbers) in
* given the string. %% is a %. %ab is the 8-bit hexadecimal number "ab"
*/
+ static inline int
+ fromhex(char ch)
+ {
+ if (ch >= '0' && ch <= '9')
+ return ch - '0';
+ if (ch >= 'a' && ch <= 'f')
+ return ch - 'a' + 10;
+ if (ch >= 'A' && ch <= 'F')
+ return ch - 'A' + 10;
+ return -1;
+ }
+
void
! rfc1738_unescape(char *s_)
{
! /* char hexnum[3]; */
! unsigned char *s = (unsigned char *) s_;
int i, j; /* i is write, j is read */
! /* unsigned int x; */
for (i = j = 0; s[j]; i++, j++) {
s[i] = s[j];
! if (s[j] != '%') {
! /* normal case, nothing more to do */
! } else if (s[j + 1] == '%') { /* %% case */
! j++; /* Skip % */
! } else {
! /* decode */
! int v1, v2, x;
! v1 = fromhex(s[j + 1]);
! if (v2 < 0)
! continue; /* non-hex or \0 */
! v2 = fromhex(s[j + 2]);
! if (v2 < 0)
! continue; /* non-hex or \0 */
! /* fromhex returns -1 on error which brings this out of range (|, not +) */
! x = v1 << 4 | v2;
! if (x > 0 && x <= 255) {
! s[i] = x;
j += 2;
}
}

*** src/cache_cf.c 2010-02-16 18:21:51.000000000 +0700
--- src/cache_cf.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 489,494 ****
--- 489,500 ----
break;
}
for (R = Config.Refresh; R; R = R->next) {
+ if (!R->flags.ignore_must_revalidate)
+ continue;
+ debug(22, 1) ("WARNING: use of 'ignore-must-revalidate' in 'refresh_pattern' violates HTTP\n");
+ break;
+ }
+ for (R = Config.Refresh; R; R = R->next) {
if (R->stale_while_revalidate <= 0)
continue;
debug(22, 1) ("WARNING: use of 'stale-while-revalidate' in 'refresh_pattern' violates HTTP\n");
***************
*** 2261,2266 ****
--- 2267,2274 ----
storeAppendPrintf(entry, " ignore-private");
if (head->flags.ignore_auth)
storeAppendPrintf(entry, " ignore-auth");
+ if (head->flags.ignore_must_revalidate)
+ storeAppendPrintf(entry, " ignore-must-revalidate");
if (head->stale_while_revalidate > 0)
storeAppendPrintf(entry, " stale-while-revalidate=%d", head->stale_while_revalidate);
#endif
***************
*** 2293,2298 ****
--- 2301,2307 ----
int ignore_no_cache = 0;
int ignore_no_store = 0;
int ignore_private = 0;
+ int ignore_must_revalidate = 0;
int ignore_auth = 0;
#endif
int stale_while_revalidate = -1;
***************
*** 2338,2343 ****
--- 2347,2354 ----
ignore_private = 1;
else if (!strcmp(token, "ignore-auth"))
ignore_auth = 1;
+ else if (!strcmp(token, "ignore-must-revalidate"))
+ ignore_must_revalidate = 1;
else if (!strcmp(token, "reload-into-ims")) {
reload_into_ims = 1;
refresh_nocache_hack = 1;
***************
*** 2397,2402 ****
--- 2408,2415 ----
t->flags.ignore_no_store = 1;
if (ignore_private)
t->flags.ignore_private = 1;
+ if (ignore_must_revalidate)
+ t->flags.ignore_must_revalidate = 1;
if (ignore_auth)
t->flags.ignore_auth = 1;
#endif

*** src/cf.data.pre 2010-03-25 21:25:33.000000000 +0700
--- src/cf.data.pre 2010-07-09 08:31:41.000000000 +0700
***************
*** 3125,3130 ****
--- 3125,3131 ----
ignore-reload
ignore-no-cache
ignore-no-store
+ ignore-must-revalidate
ignore-private
ignore-auth
stale-while-revalidate=NN
***************
*** 3164,3169 ****
--- 3165,3175 ----
from a server, only a client, though plenty of servers
send it anyway.

+ ignore-must-revalidate ignores any ``Cache-Control: must-revalidate''
+ headers received from a server. Doing this VIOLATES
+ the HTTP standard. Enabling this feature could make you
+ liable for problems which it causes.
+
ignore-private ignores any ``Cache-control: private''
headers received from a server. Doing this VIOLATES
the HTTP standard. Enabling this feature could make you

*** src/client_side_async_refresh.c 2010-05-20 16:19:09.000000000 +0700
--- src/client_side_async_refresh.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 76,81 ****
--- 76,83 ----
accessLogLog(&al, ch);
aclChecklistFree(ch);
storeClientUnregister(async->sc, async->entry, async);
+ storeUnlockObject(async->entry->mem_obj->old_entry);
+ async->entry->mem_obj->old_entry = NULL;
storeUnlockObject(async->entry);
storeUnlockObject(async->old_entry);
requestUnlink(async->request);
***************
*** 129,134 ****
--- 131,138 ----
async->entry = storeCreateEntry(url,
request->flags,
request->method);
+ if (request->store_url)
+ storeEntrySetStoreUrl(async->entry, request->store_url);
async->entry->mem_obj->old_entry = async->old_entry;
storeLockObject(async->entry->mem_obj->old_entry);
async->sc = storeClientRegister(async->entry, async);

*** src/client_side.c 2010-07-08 14:22:01.000000000 +0700
--- src/client_side.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 975,980 ****
--- 975,1005 ----
return 1;
}

+ /*
+ * Internal helper function for If-(None-)Match logics
+ */
+ static int
+ checkIfMatch(request_t * request, MemObject * mem, http_hdr_type hdr)
+ {
+ String req_etags;
+ const char *rep_etag;
+ int etag_match = 0;
+
+ if (mem->reply->sline.status != HTTP_OK) {
+ debug(33, 4) ("checkIfMatch: Reply code %d != 200\n",
+ mem->reply->sline.status);
+ return -1; /* Can't check */
+ }
+ rep_etag = httpHeaderGetStr(&mem->reply->header, HDR_ETAG);
+ req_etags = httpHeaderGetList(&request->header, hdr);
+ if (rep_etag)
+ etag_match = strListIsMember(&req_etags, rep_etag, ',');
+ if (!etag_match)
+ etag_match = strListIsMember(&req_etags, "*", ',');
+ stringClean(&req_etags);
+ return etag_match;
+ }
+
/*
* clientCacheHit should only be called until the HTTP reply headers
* have been parsed. Normally this should be a single call, but
***************
*** 1098,1142 ****
return;
}
if (httpHeaderHas(&r->header, HDR_IF_MATCH)) {
! const char *rep_etag = httpHeaderGetStr(&e->mem_obj->reply->header, HDR_ETAG);
! int has_etag = 0;
! if (rep_etag) {
! String req_etags = httpHeaderGetList(&http->request->header, HDR_IF_MATCH);
! has_etag = strListIsMember(&req_etags, rep_etag, ',');
! stringClean(&req_etags);
! }
! if (!has_etag) {
/* The entity tags does not match. This cannot be a hit for this object.
* Query the origin to see what should be done.
*/
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
}
if (httpHeaderHas(&r->header, HDR_IF_NONE_MATCH)) {
! String req_etags;
! const char *rep_etag = httpHeaderGetStr(&e->mem_obj->reply->header, HDR_ETAG);
! int has_etag;
! if (mem->reply->sline.status != HTTP_OK) {
! debug(33, 4) ("clientCacheHit: Reply code %d != 200\n",
! mem->reply->sline.status);
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
! if (rep_etag) {
! req_etags = httpHeaderGetList(&http->request->header, HDR_IF_NONE_MATCH);
! has_etag = strListIsMember(&req_etags, rep_etag, ',');
! stringClean(&req_etags);
! if (has_etag) {
! debug(33, 4) ("clientCacheHit: If-None-Match matches\n");
! if (is_modified == -1)
! is_modified = 0;
! } else {
! debug(33, 4) ("clientCacheHit: If-None-Match mismatch\n");
! is_modified = 1;
! }
}
}
if (r->flags.ims && mem->reply->sline.status == HTTP_OK) {
--- 1123,1156 ----
return;
}
if (httpHeaderHas(&r->header, HDR_IF_MATCH)) {
! int etag_match = checkIfMatch(r, mem, HDR_IF_MATCH);
!
! if (etag_match != 1) {
/* The entity tags does not match. This cannot be a hit for this object.
* Query the origin to see what should be done.
*/
+ + debug(33, 4) ("clientCacheHit: If-Match mismatch\n");
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
}
if (httpHeaderHas(&r->header, HDR_IF_NONE_MATCH)) {
! int etag_match = checkIfMatch(r, mem, HDR_IF_NONE_MATCH);
!
! if (etag_match == -1) {
! debug(33, 4) ("clientCacheHit: If-None-Match failure\n");
http->log_type = LOG_TCP_MISS;
clientProcessMiss(http);
return;
}
! if (etag_match) {
! debug(33, 4) ("clientCacheHit: If-None-Match matches\n");
! if (is_modified == -1)
! is_modified = 0;
! } else {
! debug(33, 4) ("clientCacheHit: If-None-Match mismatch\n");
! is_modified = 1;
}
}
if (r->flags.ims && mem->reply->sline.status == HTTP_OK) {
***************
*** 1156,1175 ****
* where the redirect is not explicitly as uncachable.
* Deny looping here and do not cache the response.
*/
- #if 0
/*
* XXX strcmp() sucks but the strings are both C strings. Look at String'ifying it
* XXX soon!
*/
! if (mem->reply->sline.status >= 300 && mem->reply->sline.status < 400) {
! if (!strcmp(http->uri, httpHeaderGetStr(&e->mem_obj->reply->header, HDR_LOCATION))) {
! debug(33, 1) ("clientCacheHit: Redirect Loop Detected: %s\n",http->uri);
! http->log_type = LOG_TCP_MISS;
! clientProcessMiss(http);
! return;
! }
! }
! #endif
stale = refreshCheckHTTPStale(e, r);
debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned %d\n", stale);
if (stale == 0) {
--- 1170,1188 ----
* where the redirect is not explicitly as uncachable.
* Deny looping here and do not cache the response.
*/
/*
* XXX strcmp() sucks but the strings are both C strings. Look at String'ifying it
* XXX soon!
*/
! if (mem->reply->sline.status >= 300 && mem->reply->sline.status < 400) {
! if (httpHeaderHas(&e->mem_obj->reply->header, HDR_LOCATION))
! if (!strcmp(http->uri,httpHeaderGetStr(&e->mem_obj->reply->header, HDR_LOCATION))) {
! debug(33, 2) ("clientCacheHit: Redirect Loop Detected: %s\n",http->uri);
! http->log_type = LOG_TCP_MISS;
! clientProcessMiss(http);
! return;
! }
! }
stale = refreshCheckHTTPStale(e, r);
debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned %d\n", stale);
if (stale == 0) {

*** src/client_side_etag.c 2010-02-14 14:34:59.000000000 +0700
--- src/client_side_etag.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 63,69 ****
if (etag && vary) {
char *str;
str = stringDupToC(&request->vary_encoding);
! storeAddVary(url, entry->mem_obj->method, NULL, httpHeaderGetStr(&rep->header, HDR_ETAG), request->vary_hdr, request->vary_headers, str);
safe_free(str);
}
}
--- 63,69 ----
if (etag && vary) {
char *str;
str = stringDupToC(&request->vary_encoding);
! storeAddVary(entry->mem_obj->store_url, entry->mem_obj->url, entry->mem_obj->method, NULL, httpHeaderGetStr(&rep->header, HDR_ETAG), request->vary_hdr, request->vary_headers, str);
safe_free(str);
}
}

*** src/client_side_ims.c 2010-02-19 14:57:27.000000000 +0700
--- src/client_side_ims.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 164,175 ****
/* If the ETag matches the clients If-None-Match, then return
* the servers 304 reply
*/
! if (httpHeaderHas(&new_entry->mem_obj->reply->header, HDR_ETAG) &&
! httpHeaderHas(&request->header, HDR_IF_NONE_MATCH)) {
! const char *etag = httpHeaderGetStr(&new_entry->mem_obj->reply->header, HDR_ETAG);
String etags = httpHeaderGetList(&request->header, HDR_IF_NONE_MATCH);
! int etag_match = strListIsMember(&etags, etag, ',');
stringClean(&etags);
if (etag_match) {
debug(33, 5) ("clientGetsOldEntry: NO, client If-None-Match\n");
return 0;
--- 164,182 ----
/* If the ETag matches the clients If-None-Match, then return
* the servers 304 reply
*/
! if (httpHeaderHas(&request->header, HDR_IF_NONE_MATCH)) {
String etags = httpHeaderGetList(&request->header, HDR_IF_NONE_MATCH);
! int etag_match = 0;
!
! if (httpHeaderHas(&new_entry->mem_obj->reply->header, HDR_ETAG)) {
! const char *etag = httpHeaderGetStr(&new_entry->mem_obj->reply->header, HDR_ETAG);
! etag_match = strListIsMember(&etags, etag, ',');
! }
! if (!etag_match && strListIsMember(&etags, "*", ','))
! etag_match = 1;
!
stringClean(&etags);
+
if (etag_match) {
debug(33, 5) ("clientGetsOldEntry: NO, client If-None-Match\n");
return 0;

*** src/fs/coss/store_io_coss.c 2009-07-27 10:14:38.000000000 +0700
--- src/fs/coss/store_io_coss.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 157,166 ****

/* Check to see if we need to allocate a membuf to start */
if (cs->current_membuf == NULL) {
! if (cs->curstripe < (cs->numstripes - 1))
! newmb = storeCossCreateMemBuf(SD, cs->curstripe + 1, checkf, &coll);
! else
! newmb = storeCossCreateMemBuf(SD, 0, checkf, &coll);

cs->current_membuf = newmb;
if (newmb == NULL) {
--- 157,166 ----

/* Check to see if we need to allocate a membuf to start */
if (cs->current_membuf == NULL) {
! int nextstripe = cs->curstripe + 1;
! if (nextstripe >= cs->numstripes)
! nextstripe = 0;
! newmb = storeCossCreateMemBuf(SD, nextstripe, checkf, &coll);

cs->current_membuf = newmb;
if (newmb == NULL) {
***************
*** 169,215 ****
}
cs->current_offset = cs->current_membuf->diskstart;

! /* Check if we have overflowed the disk .. */
! } else if ((cs->current_offset + allocsize) > ((off_t) SD->max_size << 10)) {
! /*
! * tried to allocate past the end of the disk, so wrap
! * back to the beginning
! */
! coss_stats.disk_overflows++;
! cs->current_membuf->flags.full = 1;
! cs->numfullstripes++;
! cs->current_membuf->diskend = cs->current_offset;
! storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
! /* cs->current_membuf may be invalid at this point */
! cs->current_offset = 0; /* wrap back to beginning */
! debug(79, 2) ("storeCossAllocate: %s: wrap to 0\n", stripePath(SD));
!
! newmb = storeCossCreateMemBuf(SD, 0, checkf, &coll);
! cs->current_membuf = newmb;
! if (newmb == NULL) {
! cs->sizerange_max = SD->max_objsize;
! return -1;
! }
! /* Check if we have overflowed the MemBuf */
! } else if ((cs->current_offset + allocsize) >= cs->current_membuf->diskend) {
/*
* Skip the blank space at the end of the stripe. start over.
*/
coss_stats.stripe_overflows++;
cs->current_membuf->flags.full = 1;
cs->numfullstripes++;
! cs->current_offset = cs->current_membuf->diskend;
storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
/* cs->current_membuf may be invalid at this point */
debug(79, 3) ("storeCossAllocate: %s: New offset - %" PRId64 "\n", stripePath(SD),
(int64_t) cs->current_offset);
! assert(cs->curstripe < (cs->numstripes - 1));
! newmb = storeCossCreateMemBuf(SD, cs->curstripe + 1, checkf, &coll);
cs->current_membuf = newmb;
if (newmb == NULL) {
cs->sizerange_max = SD->max_objsize;
return -1;
}
}
/* If we didn't get a collision, then update the current offset and return it */
if (coll == 0) {
--- 169,211 ----
}
cs->current_offset = cs->current_membuf->diskstart;

! } else if ((cs->current_offset + allocsize) > cs->current_membuf->diskend) {
/*
* Skip the blank space at the end of the stripe. start over.
*/
+ int nextstripe = cs->curstripe + 1;
coss_stats.stripe_overflows++;
cs->current_membuf->flags.full = 1;
cs->numfullstripes++;
! /* Check if we have overflowed the disk .. */
! if (nextstripe >= cs->numstripes) {
! /*
! * tried to allocate past the end of the disk, so wrap
! * back to the beginning
! */
! debug(79, 2) ("storeCossAllocate: %s: wrap to 0\n", stripePath(SD));
! nextstripe = 0; /* wrap back to beginning */
! coss_stats.disk_overflows++;
! #if LOOKS_WRONG
! /* Original disk wrap code also had this, but looks wrong to
! * me as it leaves garbage at the end of the disk. Either we
! * should do it in both cases, or not at all
! */
! cs->current_membuf->diskend = cs->current_offset;
! #endif
! }
storeCossMaybeWriteMemBuf(SD, cs->current_membuf);
/* cs->current_membuf may be invalid at this point */
debug(79, 3) ("storeCossAllocate: %s: New offset - %" PRId64 "\n", stripePath(SD),
(int64_t) cs->current_offset);
! assert(nextstripe <>numstripes);
! newmb = storeCossCreateMemBuf(SD, nextstripe, checkf, &coll);
cs->current_membuf = newmb;
if (newmb == NULL) {
cs->sizerange_max = SD->max_objsize;
return -1;
}
+ cs->current_offset = cs->current_membuf->diskstart;
}
/* If we didn't get a collision, then update the current offset and return it */
if (coll == 0) {

*** src/http.c 2010-04-20 12:01:43.000000000 +0700
--- src/http.c 2010-07-09 08:31:42.000000000 +0700
***************
*** 334,352 ****
*/
if (!refreshIsCachable(httpState->entry) && !REFRESH_OVERRIDE(store_stale))
return 0;
- /* don't cache objects from peers w/o LMT, Date, or Expires */
- /* check that is it enough to check headers @?@ */
- if (rep->date > -1)
- return 1;
- else if (rep->last_modified > -1)
- return 1;
- else if (!httpState->peer)
- return 1;
- /* @?@ (here and 302): invalid expires header compiles to squid_curtime */
- else if (rep->expires > -1)
- return 1;
else
! return 0;
/* NOTREACHED */
break;
/* Responses that only are cacheable if the server says so */
--- 334,341 ----
*/
if (!refreshIsCachable(httpState->entry) && !REFRESH_OVERRIDE(store_stale))
return 0;
else
! return 1;
/* NOTREACHED */
break;
/* Responses that only are cacheable if the server says so */

***************
*** 926,932 ****
*/
if (len > 0 && httpState->chunk_size == 0) {
if (Config.onoff.log_http_violations)
! debug(11, 1) ("httpReadReply: Unexpected reply body data from \"%s %s\"\n",
urlMethodGetConstStr(orig_request->method), storeUrl(entry));
comm_close(fd);
return;
--- 926,933 ----
*/
if (len > 0 && httpState->chunk_size == 0) {
if (Config.onoff.log_http_violations)
! debug(11, Config.onoff.relaxed_header_parser <= 0 || keep_alive ? 1 : 2)
! ("httpReadReply: Unexpected reply body data from \"%s %s\"\n",
urlMethodGetConstStr(orig_request->method), storeUrl(entry));
comm_close(fd);
return;
*** src/refresh.c 2010-04-10 13:40:41.000000000 +0700
--- src/refresh.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 274,281 ****
debug(22, 3) ("\tcheck_time:\t%s\n", mkrfc1123(check_time));
debug(22, 3) ("\tentry->timestamp:\t%s\n", mkrfc1123(entry->timestamp));

! if (EBIT_TEST(entry->flags, ENTRY_REVALIDATE) && staleness > -1) {
! debug(22, 3) ("refreshCheck: YES: Must revalidate stale response\n");
return STALE_MUST_REVALIDATE;
}
/* request-specific checks */
--- 274,285 ----
debug(22, 3) ("\tcheck_time:\t%s\n", mkrfc1123(check_time));
debug(22, 3) ("\tentry->timestamp:\t%s\n", mkrfc1123(entry->timestamp));

! if (EBIT_TEST(entry->flags, ENTRY_REVALIDATE) && staleness > -1
! #if HTTP_VIOLATIONS
! && !R->flags.ignore_must_revalidate
! #endif
! ) {
! debug(22, 3) ("refreshCheck: YES: Must revalidate stale response\n");
return STALE_MUST_REVALIDATE;
}
/* request-specific checks */
***************
*** 334,339 ****
--- 338,353 ----
* At this point the response is stale, unless one of
* the override options kicks in.
*/
+ #if HTTP_VIOLATIONS
+ if (sf.expires && R->flags.override_expire && age <>min) {
+ debug(22, 3) ("refreshCheck: NO: age < min && override-expire\n");
+ return FRESH_OVERRIDE_EXPIRES;
+ }
+ if (sf.lmfactor && R->flags.override_lastmod && age <>min) {
+ debug(22, 3) ("refreshCheck: NO: age < min && override-lastmod\n");
+ return FRESH_OVERRIDE_LASTMOD;
+ }
+ #endif
if (entry->mem_obj) {
int stale_while_revalidate = -1;
if (entry->mem_obj->reply && entry->mem_obj->reply->cache_control && EBIT_TEST(entry->mem_obj->reply->cache_control->mask, CC_STALE_WHILE_REVALIDATE))
***************
*** 357,382 ****
if (delta < 0 && staleness + delta < 0) {
return STALE_WITHIN_DELTA;
}
! if (sf.expires) {
! #if HTTP_VIOLATIONS
! if (R->flags.override_expire && age <>min) {
! debug(22, 3) ("refreshCheck: NO: age < min && override-expire\n");
! return FRESH_OVERRIDE_EXPIRES;
! }
! #endif
return STALE_EXPIRES;
- }
if (sf.max)
return STALE_MAX_RULE;
! if (sf.lmfactor) {
! #if HTTP_VIOLATIONS
! if (R->flags.override_lastmod && age <>min) {
! debug(22, 3) ("refreshCheck: NO: age < min && override-lastmod\n");
! return FRESH_OVERRIDE_LASTMOD;
! }
! #endif
return STALE_LMFACTOR_RULE;
- }
return STALE_DEFAULT;
}

--- 371,382 ----
if (delta < 0 && staleness + delta < 0) {
return STALE_WITHIN_DELTA;
}
! if (sf.expires)
return STALE_EXPIRES;
if (sf.max)
return STALE_MAX_RULE;
! if (sf.lmfactor)
return STALE_LMFACTOR_RULE;
return STALE_DEFAULT;
}

*** src/store.c 2010-03-06 09:36:15.000000000 +0700
--- src/store.c 2010-07-09 08:31:41.000000000 +0700
***************
*** 553,559 ****
stringClean(&varyhdr);
#endif
str = stringDupToC(&vary);
! storeAddVary(mem->url, mem->method, newkey, httpHeaderGetStr(&mem->reply->header, HDR_ETAG), str, mem->vary_headers, mem->vary_encoding);
safe_free(str);
stringClean(&vary);
}
--- 553,559 ----
stringClean(&varyhdr);
#endif
str = stringDupToC(&vary);
! storeAddVary(mem->store_url, mem->url, mem->method, newkey, httpHeaderGetStr(&mem->reply->header, HDR_ETAG), str, mem->vary_headers, mem->vary_encoding);
safe_free(str);
stringClean(&vary);
}

*** src/store_client.c 2009-04-21 15:37:50.000000000 +0700
--- src/store_client.c 2010-11-24 09:51:46.000000000 +0700
***************
*** 894,905 ****
debug(20, 3) ("CheckQuickAbort2: YES !mem->request->flags.cachable\n");
return 1;
}
if (EBIT_TEST(entry->flags, KEY_PRIVATE)) {
debug(20, 3) ("CheckQuickAbort2: YES KEY_PRIVATE\n");
return 1;
}
- expectlen = mem->reply->content_length + mem->reply->hdr_sz;
- curlen = mem->inmem_hi;
minlen = Config.quickAbort.min << 10;
if (minlen < 0) {
debug(20, 3) ("CheckQuickAbort2: NO disabled\n");
--- 894,909 ----
debug(20, 3) ("CheckQuickAbort2: YES !mem->request->flags.cachable\n");
return 1;
}
+ expectlen = httpReplyBodySize(mem->method, mem->reply) + mem->reply->hdr_sz;
+ curlen = mem->inmem_hi;
+ if (expectlen == curlen) {
+ debug(20, 3) ("CheckQuickAbort2: NO already finished\n");
+ return 0;
+ }
if (EBIT_TEST(entry->flags, KEY_PRIVATE)) {
debug(20, 3) ("CheckQuickAbort2: YES KEY_PRIVATE\n");
return 1;
}
minlen = Config.quickAbort.min << 10;
if (minlen < 0) {
debug(20, 3) ("CheckQuickAbort2: NO disabled\n");

*** src/store_vary.c 2010-10-19 09:22:25.000000000 +0700
--- src/store_vary.c 2010-11-24 12:09:37.000000000 +0700
***************
*** 6,11 ****
--- 6,12 ----
StoreEntry *oe;
StoreEntry *e;
store_client *sc;
+ char *store_url;
char *url;
char *key;
char *vary_headers;
***************
*** 57,62 ****
--- 58,64 ----
storeUnlockObject(state->oe);
state->oe = NULL;
}
+ safe_free(state->store_url);
safe_free(state->url);
safe_free(state->key);
safe_free(state->vary_headers);
***************
*** 312,318 ****
* At leas one of key or etag must be specified, preferably both.
*/
void
! storeAddVary(const char *url, method_t * method, const cache_key * key, const char *etag, const char *vary, const char *vary_headers, const char *accept_encoding)
{
AddVaryState *state;
request_flags flags = null_request_flags;
--- 314,320 ----
* At leas one of key or etag must be specified, preferably both.
*/
void
! storeAddVary(const char *store_url, const char *url, method_t * method, const cache_key * key, const char *etag, const char *vary, const char *vary_headers, const char *accept_encoding)
{
AddVaryState *state;
request_flags flags = null_request_flags;
***************
*** 326,332 ****
state->accept_encoding = xstrdup(accept_encoding);
if (etag)
state->etag = xstrdup(etag);
! state->oe = storeGetPublic(url, method);
debug(11, 2) ("storeAddVary: %s (%s) %s %s\n",
state->url, state->key, state->vary_headers, state->etag);
if (state->oe)
--- 328,334 ----
state->accept_encoding = xstrdup(accept_encoding);
if (etag)
state->etag = xstrdup(etag);
! state->oe = storeGetPublic(store_url ? store_url : url, method);
debug(11, 2) ("storeAddVary: %s (%s) %s %s\n",
state->url, state->key, state->vary_headers, state->etag);
if (state->oe)

*** src/store_vary.h 2010-02-14 14:34:59.000000000 +0700
--- src/store_vary.h 2010-07-09 08:31:42.000000000 +0700
***************
*** 4,10 ****
extern void storeLocateVaryDone(VaryData * data);
extern void storeLocateVary(StoreEntry * e, int offset, const char *vary_data,
String accept_encoding, STLVCB * callback, void *cbdata);
! extern void storeAddVary(const char *url, method_t * method, const cache_key * key,
const char *etag, const char *vary, const char *vary_headers,
const char *accept_encoding);

--- 4,10 ----
extern void storeLocateVaryDone(VaryData * data);
extern void storeLocateVary(StoreEntry * e, int offset, const char *vary_data,
String accept_encoding, STLVCB * callback, void *cbdata);
! extern void storeAddVary(const char *store_url, const char *url, method_t * method, const cache_key * key,
const char *etag, const char *vary, const char *vary_headers,
const char *accept_encoding);

*** src/structs.h 2010-04-21 21:10:06.000000000 +0700
--- src/structs.h 2010-07-09 08:31:42.000000000 +0700
***************
*** 1706,1712 ****
unsigned int reload_into_ims:1;
unsigned int ignore_reload:1;
unsigned int ignore_no_cache:1;
! unsigned int ignore_no_store:1;
unsigned int ignore_private:1;
unsigned int ignore_auth:1;
#endif
--- 1706,1713 ----
unsigned int reload_into_ims:1;
unsigned int ignore_reload:1;
unsigned int ignore_no_cache:1;
! unsigned int ignore_no_store:1;
! unsigned int ignore_must_revalidate:1;
unsigned int ignore_private:1;
unsigned int ignore_auth:1;
#endif

*** src/main.c 2010-04-21 21:10:06.000000000 +0700
--- src/main.c 2010-07-12 09:35:25.444745821 +0700
***************
*** 581,587 ****

_db_init(Config.debugOptions);
_db_init_log(Config.Log.log);
! fd_open(fileno(debug_log), FD_LOG, Config.Log.log);
#if MEM_GEN_TRACE
log_trace_init("/tmp/squid.alloc");
#endif
--- 581,588 ----

_db_init(Config.debugOptions);
_db_init_log(Config.Log.log);
! if (debug_log != stderr)
! fd_open(fileno(debug_log), FD_LOG, Config.Log.log);
#if MEM_GEN_TRACE
log_trace_init("/tmp/squid.alloc");
#endif

*** src/main.c 2010-04-21 21:10:06.000000000 +0700
--- src/main.c 2010-07-15 09:08:53.385643957 +0700
***************
*** 682,688 ****
#if USE_WCCPv2
wccp2Init();
#endif
! serverConnectionsOpen();
neighbors_init();
if (Config.chroot_dir)
no_suid();
--- 682,689 ----
#if USE_WCCPv2
wccp2Init();
#endif
! if (!opt_foreground_rebuild)
! serverConnectionsOpen();
neighbors_init();
if (Config.chroot_dir)
no_suid();
***************
*** 911,916 ****
--- 912,922 ----
#endif
serverConnectionsClose();
eventAdd("SquidShutdown", SquidShutdown, NULL, (double) (wait + 1), 1);
+ } else if (opt_foreground_rebuild && !store_dirs_rebuilding) {
+ opt_foreground_rebuild = 0;
+ enter_suid();
+ serverConnectionsOpen();
+ leave_suid();
}
/* Set a maximum loop delay; it'll be lowered elsewhere as appropriate */
loop_delay = 60000;

*** src/client_side_request_parse.c 2010-04-05 14:40:47.000000000 +0700
--- src/client_side_request_parse.c 2010-07-27 11:01:51.520075515 +0700
***************
*** 516,521 ****
--- 516,569 ----
ret = -1;
goto finish;
}
+
+ /*
+ * Normalize Request Cache-Control / If-Modified-Since Headers
+ * Don't let client by-pass the cache if there is cached content.
+ */
+ if(httpHeaderHas(&request->header,HDR_CACHE_CONTROL)) {
+ httpHeaderDelByName(&request->header,"cache-control");
+ }
+
+ /*
+ * Un-comment this if you want Squid to always respond with the request
+ * instead of returning back with a 304 if the cache has not changed.
+ */
+
+ if(httpHeaderHas(&request->header,HDR_IF_MODIFIED_SINCE)) {
+ httpHeaderDelByName(&request->header,"if-modified-since");
+ }
+
+ /*
+ * Normalize Accept-Encoding Headers sent from client
+ */
+ if(httpHeaderHas(&request->header,HDR_ACCEPT_ENCODING)) {
+ String val = httpHeaderGetByName(&request->header,"accept-encoding");
+ if(val.buf) {
+ if(strstr(val.buf,"gzip") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ httpHeaderPutStr(&request->header,HDR_ACCEPT_ENCODING,"gzip");
+ } else if(strstr(val.buf,"deflate") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ httpHeaderPutStr(&request->header,HDR_ACCEPT_ENCODING,"deflate");
+ } else {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ }
+ }
+ stringClean(&val);
+ }
+
+ /*
+ * Normalize Accept-Encoding Headers for video/image content
+ */
+ char *mime_type = mimeGetContentType(http->uri);
+ if(mime_type) {
+ if(strstr(mime_type,"image") != NULL || strstr(mime_type,"video") != NULL || strstr(mime_type,"audio") != NULL) {
+ httpHeaderDelByName(&request->header,"accept-encoding");
+ }
+ }
+
+
/*
* If we read past the end of this request, move the remaining
* data to the beginning




atau bisa donlot disini

Read More......

Selasa, 27 Juli 2010

yuk mainan lusca cache di freebsd....

Setelah sekian lama "bermain-main" dengan squid 2.7 STABLE7, iseng iseng mencoba lusca cache besutan Adrian Chadd. Lusca cache sebenarnya masih kerabatan sama squid 2.7 soalnya lusca dikembangkan dari sana. Perbedaannya, lusca salah satunya mencoba menitikberatkan peningkatan performa dalam peyimpanan cache ke hardisk dan beberapa script dicoba ditata ulang untuk meningkatkan performanya. Salah satu hasil signifikan adalah Modul COSS, dimana COSS lusca proses rebuildingnya lebih cepet dibanding COSS di squid aslinya.

Ok, back to topik again..
karena lusca ini nantinya akan berjalan di freebsd (saya pake freebsd 8.0) maka sudah tentu harus sukses dulu install freebsdnya :D
Untuk hanya menjalankan lusca cache difreebsd, paket yang dibutuhkan hanya perl. Anda bisa menginstallnya via port. Agar lebih "menyenangkan" , segala proses install menggunakan putty dan winscp jadi pengerjaannya via remote dari komputer/laptop basis win***s.

1. install perl via port

#cd /usr/ports/lang/perl5.10
#make install clean


tunggu beberapa saat, untuk mengetest apakah perl sudah terinstall ketikan saja diterminal perl -v

2. Download source lusca cache.

Source lusca cache versi terakhir bisa di donlot di code.google.com/p/lusca-cache/. Setelah di donlot silahkan di trasnfer ke mesin freebsd memakai winscp. Anda bisa menaruhnya di /usr/local/src
donlot juga patch ini lusca-patch nanti diesktrak dan taruh juga di /usr/local/src

3. Kompilasi lusca cache.

Setelah di transfer ke mesin freebsd via winscp dan diletakan di /usr/local/src selanjutnya kita unpack source dan lakukan patch :

#cd /usr/local/src
#tar -xvf LUSCA_HEAD-rxxxx.tar.gz
#cd LUSCA_HEAD-rxxxx
#patch -p0 < ../lusca-r14723-sum14rdi.patch #./configure --bindir=/usr/local/bin --sbindir=/usr/local/sbin --sysconfdir=/usr/local/etc/squid --datadir=/usr/local/etc/squid --libexecdir=/usr/local/libexec/squid --localstatedir=/var/log/squid --enable-removal-policies="heap" --enable-auth="basic ntlm digest" --enable-digest-auth-helpers=password --with-pthreads --enable-async-io=24 --with-aufs-threads=24 --enable-storeio="aufs coss" --disable-ident-lookups --enable-delay-pools --enable-snmp --enable-cache-digests --disable-wccp --enable-useragent-log --enable-http-violations --enable-arp-acl --enable-pf-transparent --disable-follow-x-forwarded-for --with-large-files --enable-large-cache-files --enable-default-err-language=English #make && make install

4. Running for first time

sebelum squid dijalankan pastikan telh dibuat user dan group squid di freebsd, kemudian squid.conf yang ada di /usr/local/etc/squid diconfigure sesuai kebutuhannya. Anda bisa mencontoh squid.conf saya dan pastikan anda untuk menyesuaikan dengan kondisi jaringan anda.
Jangan lupa untuk mengubah kepemilikan directory untuk cache kepada squid. Setelah itu baru :

# squid -z
# squid -DF

agar squid dapat jalan otomatis ketika restart, maka taruh squid -DF di file /etc/rc.local, jika tidak ada maka silahkan membuatnya.

SELESAI






Read More......

Install FreeBSD ke hardisk (bukan tutorial)

Setelah sekian lama gak tulis-tulis (akibat lupa username dan password ke blog ini) sekarang mencoba menuangkan kembali pengalaman saya. Ini adalah yang pertama pasca ketemunya username dan passwordnya :D

Sebenarnya sudah banyak yang mengulas bagaimana cara menginstall freebsd ke hardisk. Anda bisa "sowan" ke simbah google, salah dua nya yang menurut saya simple ada disini

1. www.freebsd.org
2. freebsd.or.id

secara garis besar menginstall freebsd ke hardisk meliputi beberapa bagian besar yaitu:
1. pemilihan setup
2. pemilihan partisi.
3. pemilihan file system & swap (labeling)
4. pemilihan source
5. memulai proses instalasi

 

Read More......

Selasa, 08 Desember 2009

(Ter) Gila2 sama Linux Mint 7

Sudah sekitar 3 bulan ini lagi berusaha untuk pindah dari lingkungan Microsoft Windows XP ke Linux Mint 7. Udah Bosen "menggauli" MW XP (karena gak punya duit buat beli yang asli) jadinya cari yang gratisan.
Dipilih Linux Mint 7 karena saya masih anak TK untuk urusan Linux dan merasa ini yang paling cocok buat saya. Walau harus dibayar dengan hardware yang harus diatas rata2 jika menggunakan linux yang lain.
Linux Mint 7 merupakan turunan dari ubuntu 9.04 alias Jaunty, jadi kita juga bisa memakai repo dari ubuntu Jaunty jika ingin menginstall software.
Yang bikin senengnya adalah hampir semua hardware yang ada dikomputer saya dapat berfungsi seperti ketika di MW XP seperti TV Tuner Pixelview, Printer HP 1020 Scanner HP 4070.

Read More......

Kamis, 02 Juli 2009

kern.maxfiles di freebsd

Kejadian berawal dari perubahan settingan untuk menurunkan jumlah file deskriptor di squid yang berjalan disystem freebsd yang semula 8192 menjadi 1082. Setelah googling sana sini didapat caranya yaitu dengan merubah angka kern.maxfiles dan kern.maxfilesperproc yang ditaruh di /boot/loader.conf
maksudnya menurunkan file deskriptor squid, dari hasil googling katanya gak perlu besar-besar nilainya jika kebesaran malah membebani squidnya.
singkat kata nilai kern.maxfiles dan kern.maxfilesperproc dikasih nilai 1024 sehingga didapat nilai file deskriptor sebesar 1082, lalu komputer squidnya di restart.
karena waktu itu pengerjaannya melalui proses remote, dimana yang dioprek adalah proxy warnet dan saya ada dikantor jadinya gak tahu efeknya terhadap browsing. selang 10 menit operator warnetnya complain via ym katanya browsingnya jadi lelet.
sebetulnya saya tidak tahu klo leletnya browsing diakibatkan oleh perubahan nilai kern.maxfiles dan kern.maxfilesperproc, karena dari hasil tail -f access.log terlihat access time nya yang besar-besar setiap ada request dari client sampai 5 digit orde time accessnya. karena access time yang lama, pikiran saya tertuju sama hardisk yang digunakan untuk simpan cache. Kebetulan hardisk untuk simpan cache memang panas sekali, lalu saya pasang kipas. Tapi tidak membawa perubahan terhadap time accessnya yang masih besar.
akhirnya setelah mentok, ya kembalikan saja settingan ke awal sewaktu berjalan normal dimana:
kern.maxfiles=32768
kern.maxfilesperproc=32768
nilai ini saya pasang sembarang, jadi silahkan bereksperimen sendiri, yang pasti dengan angka segini sudah cukup gesit buat kerja proxy saya.
sebagai tambahan hardisk yang digunakan semuanya PATA IDE berjumlah 2 buah.

Read More......

Jumat, 26 Juni 2009

udah lama euy......

udah lama gak "nabung" tulisan......
sedikit curhat masalah proxy freebsd. Ternyata perangkat keras komputer terutama hardisk benar2 mempengaruhi performance dari proxy. Hardisk yang cepet panas berdampak ke access time dari proxy jadi bertambah, efeknya ngenet jadi agak sedikit lag.
Jadi pastikan hardware terutama hardisknya sehat, sirkulasi udara di komputer yang buat proxy dipastikan juga lancar.
Walaupun diping sana sini masih lancar dan memberikan latency yang kecil, tapi jika hardisknya panas maka squidnya akan sedikit macet.

Read More......

Senin, 08 Juni 2009

Berbagi ide....bandwidth management

Pada kesempatan ini saya ingin menuangkan ide saya, dari hasil percobaan di tiga tempat. Ide mengenai pembagian bandwidth internet. Idenya adalah sebagai berikut, saya menginginkan agar aktifitas browsing mendapatkan bandwidth yang memadai tetapi tidak untuk aktifitas donlot, p2p dan watching video streaming.
Untuk keperluan tersebut saya menggunakan mikrotik dan squid proxy yang jalan di freebsd.Mikrotik digunakan sebagai pembatas global, dalam percobaan saya memberikan bandwidth donlot sebesar 256Kbps per klien dan upload saya batasi 32Kbps. Sedangkan squid disamping sebagai cache saya manfaatkan fungsi delay pools nya untuk menangkap aktifitas donlot extension tertentu. Untuk aktifitas donlot extension tertentu saya berikan speed 9 KBps atau setara 72Kbps.
Alasan mengapa untuk browsing diberikan sebesar 256Kbps karena saya beranggapan untuk aktifitas browsing kebutuhan bandwidthnya adalah fluxtuatif alias naik turun dan tidak kontinu, sedangkan untuk donlot dia bersifat kontinu. Namun untuk penggunaan browsing yang agresif mungkin nilai 256Kbps perlu diturunkan, karena walaupun fluxtuatif jika agresif maka kesannya juga seperti kontinu.

Untuk setting detailnya....menyusul, setelah didapatkan hasil yang pas...

Read More......

Jumat, 08 Mei 2009

Creative Theme Day 2009........

satu lagi nih ide dari temen-temen yang oke punya.....Creative Theme Day.....
lebih lengkapnya kesini aja ya....
http://creativethemeday.com/ctd-update/press-release-creative-theme-day-dengung-tahun-kreatif-2009-di-dunia-maya/

”Creative

ayo satukan tekad untuk indonesia yang lebih sejahtera, adil dan makmur..........amin

Read More......
 
from my XP © Template Design by Herro