Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#1753 closed new feature (fixed)

Re-integrate piwik tracking

Reported by: jsfan Owned by: jsfan
Priority: major Milestone: 1.1
Component: FrameWork Keywords:
Cc: guaka, crumbking, planetcruiser

Description (last modified by planetcruiser)

Intro:

Issue:

  • We need to put tracking code into Rox for this. This could be JS or an image depending on what we are after (cf. #1515).

Solution:

  • Add configuration for inclusion URL and site ID, add examples to default config, commented out
  • Don't show Piwik code if either on of the config settings is missing or empty (this way we are gracefully backwards compatible)
  • Give guaka access to stats, then wait for people to ask for it

Discussion:

  • Which one of the two do we use (or both and JS that disables the image if JS is activated)?

Change History (24)

comment:1 Changed 6 years ago by jsfan

  • Cc planetcruiser added

comment:2 follow-up: Changed 6 years ago by guaka

  • I don't "need" it but I would like access.
  • The tracking code should only be there on the live server.
  • JS with image fallback is best for getting proper stats.
  • Set up Piwik according to http://piwik.org/privacy (possibly create other tickets for this as there might be non-trivial parts)
Last edited 6 years ago by guaka (previous) (diff)

comment:3 in reply to: ↑ 2 ; follow-up: Changed 6 years ago by jsfan

Replying to guaka:

  • I don't "need" it but I would like access.

No problem with me. Once it's running, I'll create an account for you. :)

  • The tracking code should only be there on the live server.

Hm. Not sure about that one. File include based on config?

  • JS with image fallback is best for getting proper stats.

Yep. Even though Meinhard is also right in remarking that it might not be the right thing to spy on members as much as we could. So, JS with fallback or image only?

Actually, this is quite simple. Really it's only two settings. Not sure about the cron job though. I've set it up in /etc/cron.d/piwik as suggested but when I ran it it didn't come back for several minutes despite the DB still being empty. :( Might have to look into that again.

Last edited 6 years ago by jsfan (previous) (diff)

comment:4 Changed 6 years ago by jsfan

Ok. I think I figured out the cron job. All good on that front. Just the instructions being slightly wrong.

comment:5 Changed 6 years ago by jsfan

  • Milestone changed from 1.1 to 1.0-sec

comment:6 in reply to: ↑ 3 ; follow-up: Changed 6 years ago by planetcruiser

  • Description modified (diff)

Replying to jsfan:

  • The tracking code should only be there on the live server.

Hm. Not sure about that one. File include based on config?

we should handle alpha as a second site to monitor. that's where the config will come in handy. i am actually interested how many people surf on alpha and where they come from.

we should really password protect alpha again to hide it from search engines. but that's another issue.

  • JS with image fallback is best for getting proper stats.

Yep. Even though Meinhard is also right in remarking that it might not be the right thing to spy on members as much as we could. So, JS with fallback or image only?

shall we start with image only and then see if that delivers us enough stats? most likely the js goes up and down the DOM and does other things which consume more cpu and memory on the client.

Actually, this is quite simple. Really it's only two settings. Not sure about the cron job though. I've set it up in /etc/cron.d/piwik as suggested but when I ran it it didn't come back for several minutes despite the DB still being empty. :( Might have to look into that again.

cool. do you document this, or will it be obvious to everyone that needs to do maintenance on it in let's say 2 years?

i added some structure to the ticket description.

comment:7 in reply to: ↑ 6 Changed 6 years ago by jsfan

  • Owner set to jsfan
  • Status changed from new to assigned

Replying to planetcruiser:

Replying to jsfan:

  • The tracking code should only be there on the live server.

Hm. Not sure about that one. File include based on config?

we should handle alpha as a second site to monitor. that's where the config will come in handy. i am actually interested how many people surf on alpha and where they come from.

Sure. If the id is part of the config, that shouldn't be an issue.

we should really password protect alpha again to hide it from search engines. but that's another issue.

Wouldn't a robots.txt be enough for that?

  • JS with image fallback is best for getting proper stats.

Yep. Even though Meinhard is also right in remarking that it might not be the right thing to spy on members as much as we could. So, JS with fallback or image only?

shall we start with image only and then see if that delivers us enough stats? most likely the js goes up and down the DOM and does other things which consume more cpu and memory on the client.

I'll implement both, so we can change it with config only.

Actually, this is quite simple. Really it's only two settings. Not sure about the cron job though. I've set it up in /etc/cron.d/piwik as suggested but when I ran it it didn't come back for several minutes despite the DB still being empty. :( Might have to look into that again.

cool. do you document this, or will it be obvious to everyone that needs to do maintenance on it in let's say 2 years?

Well, all I have done is follow the instructions as found on piwik.org. Nothing fancy so far. I could document where the cronjob is but that's probably it.

comment:8 Changed 6 years ago by jsfan

Deployed on alpha.

Alpha reports as site #2, www as site #1. Currently stats are set to image only.

comment:9 Changed 6 years ago by jsfan

Maybe we also want to do this before releasing on www... http://piwik.org/log-analytics/

Also, we might want to use the GeoIP plugin, I suspect. (http://dev.piwik.org/trac/ticket/45)

comment:10 Changed 6 years ago by jsfan

This all seem to work ok. Can anyone confirm that there is working Piwik code on alpha, so we can close this one?

comment:11 Changed 6 years ago by guaka

Seems okay. But it's very slow.

comment:12 follow-up: Changed 6 years ago by guaka

It also seems that piwik is tracking its own use. That might be fine but then it should be a 3rd website!

Also, I think it's a good idea to use piwik.bewelcome.org as the URL. Would make it easier to put it on another server at some point. And there'd be fewer cookie data to go across the wire.

comment:13 Changed 6 years ago by jsfan

The speed was due to the cron job failing. I have fixed that now and it is now gernating the reports periodically.

As for the Piwik tracking its own use, this is actually not the case. The reason it shows up at the moment is because the current stats of www are based on log files and not the tracker which isn't live yet.

I don't oppose using piwik.bewelcome.org but don't think it's a big deal to change that later if necessary.

comment:14 Changed 6 years ago by jsfan

We seem to have a bigger problem there. Even with now 512M limit on PHP the cron job (which fetches the web URLs) runs out of memory on a number of tasks. :(

I've gone up to 1G for now but we might want to try going lower again.

Last edited 6 years ago by jsfan (previous) (diff)

comment:15 follow-up: Changed 6 years ago by crumbking

I remember a downtime of BW because of the downtime of piwik. How can we make sure this does not happen in future if piwik goes down?

comment:16 in reply to: ↑ 12 Changed 6 years ago by planetcruiser

Replying to guaka:

Also, I think it's a good idea to use piwik.bewelcome.org as the URL. Would make it easier to put it on another server at some point. And there'd be fewer cookie data to go across the wire.

yes, certainly a good idea, but we would need a new ssl certificate (at cost) because ssl bw will also connect to ssl piwik, right?

so we would need to run this by the board or someone in charge of finances. not sure what the process is for this

comment:17 in reply to: ↑ 15 Changed 6 years ago by planetcruiser

Replying to crumbking:

I remember a downtime of BW because of the downtime of piwik. How can we make sure this does not happen in future if piwik goes down?

i think this was because the old dev server gnat was down i think. to reasons why this can't happen again:

  • the new piwik is hosted on a proper server (same machine as live site), not an adsl line ;)
  • we are only including a 1px image this time, not js like before. if the image loads slow, the site will still load at normal speed

comment:18 Changed 6 years ago by guaka

I think piwik over ssl is overkill. But maybe when something is ssl it means it shouldn't go into piwik?

comment:19 follow-up: Changed 6 years ago by jsfan

You can use http for piwik if you like. However, I prefer using https for anything that involves a password even if there isn't much behind it and the password is unique to the service. And https is not as expensive as generally believed (cf. http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html).

Piwik with JS uses the same protocol for tracking as is used to access the site. The Piwik image will always be served as http.

comment:20 Changed 6 years ago by crumbking

  • Resolution set to fixed
  • Status changed from assigned to closed

I guess this ticket is done. If piwik works. PS: I would like to have an account. :-)

comment:21 in reply to: ↑ 19 ; follow-up: Changed 6 years ago by planetcruiser

  • Resolution fixed deleted
  • Status changed from closed to reopened

Replying to jsfan:

You can use http for piwik if you like. However, I prefer using https for anything that involves a password even if there isn't much behind it and the password is unique to the service. And https is not as expensive as generally believed (cf. http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html).

yes, allowing (or forcing?) https everywhere is covered another set of tickets. this should become a priority after the osm release.

The Piwik image will always be served as http.

this will result in "this secure page contains insecure elements" browser warnings, which we had complaints about in the past. could we make this protocol aware, too?

another thing, the image tag now reads:

<img src="http://www.bewelcome.org/piwik//piwik.php?idsite=2&amp;rec=1" style="border:0" alt="" />

-> there is a double slash (wrong base url in rox.ini?) and the image should have "width=1 height=1" to make the page render as fast as possible, even if it's still waiting for piwik to respond

comment:22 in reply to: ↑ 21 ; follow-up: Changed 6 years ago by jsfan

Replying to planetcruiser:

The Piwik image will always be served as http.

this will result in "this secure page contains insecure elements" browser warnings, which we had complaints about in the past. could we make this protocol aware, too?

I thought the same thing as I wrote my last comment. Fixed now.

another thing, the image tag now reads:

<img src="http://www.bewelcome.org/piwik//piwik.php?idsite=2&amp;rec=1" style="border:0" alt="" />

-> there is a double slash (wrong base url in rox.ini?) and the image should have "width=1 height=1" to make the page render as fast as possible, even if it's still waiting for piwik to respond

I can't confirm the double slash. I only see a single one. A trailing slash from config should be stripped automatically.

I have now added width and height. I'd pretty much only copied and pasted the Piwik code and not thought much about it (except for correcting faulty code somewhere).

comment:23 in reply to: ↑ 22 Changed 6 years ago by planetcruiser

  • Resolution set to fixed
  • Status changed from reopened to closed

Replying to jsfan:

I thought the same thing as I wrote my last comment. Fixed now.

sweet.

I can't confirm the double slash. I only see a single one. A trailing slash from config should be stripped automatically.

weird, indeed it's gone now. i swear it was there before! ;)

all looks good for me now, closing.

Note: See TracTickets for help on using tickets.