Opened 6 years ago

Closed 4 years ago

#1885 closed task (fixed)

Data retention

Reported by: globetrotter_tt Owned by: shevek
Priority: critical Milestone: 2.7
Component: BW Database Keywords: data privacy member
Cc: jsfan, planetcruiser, jeanyves, mahouni

Description

Issue:

  • We need to make sure that data retention is in line with the french (and german?) law and communicate it clearly which data will be kept when a user deletes his profile.

Current status:

  • A users change the status of his profile to "AskToLeave" -> http://www.bewelcome.org/deleteprofile
  • The data is still in the database, but the profile is blocked and not visible on the website anymore
  • A query overwrites the profile with anonymous data (***deleted***)
  • Username is changed to "RETIRED_X" where X is the member ID
  • Location of the profile is changed to "Auroville"
  • Profile picture and thumbs are kept and visible on the website
  • Private messages (received and sent) are kept and visbile in the recipient's inbox
  • Comments (received and sent) are kept but not visible on the website
  • Forum posts are kept and visibile on the website
  • Blog posts are kept and visible on the website
  • User gallery is kept and visible on the website

Fomerly discussed in:

http://www.bewelcome.org/forums/s543-Does_BeWelcome_needs_to_Keep_the_data_about_members_who_have_left

Task:

  • Check if all profile data is anonymized and no private data is kept
  • Delete all uploaded pictures from profiles with status "AskToLeave"

Change History (24)

comment:1 Changed 6 years ago by planetcruiser

  • Cc mahouni added

matthias: what is the status here? was i supposed to make sure this makes it into this or the next release? i forgot ;)

comment:2 follow-up: Changed 6 years ago by planetcruiser

  • Owner changed from planetcruiser to globetrotter_tt
  • Status changed from new to assigned

i will focus on api and welen. someone else please?

comment:3 in reply to: ↑ 2 Changed 6 years ago by mahouni

Replying to planetcruiser:

i will focus on api and welen. someone else please?

not sure, if I will have enough time in the next days. But reporting here what I've started to experiment with:

In ./build/members/members.ctrl.php in the function retireProfile() I did the following to overwrite the profile picture by a standard retired_avatar.png:

    public function retireProfile(StdClass $args, $memory, $stuff1, $stuff2)
    {
    ...
    ...
        if (isset($args->post['Complete_retire']))
        {
            $retiredimage =  HTDOCS_BASE . "images/misc/retired_avatar.png";
            $k = $this->model->avatarMake($member->getPKValue(), $retiredimage, false);
            $member->removeProfile();
        }

In the file ./build/members/members.entity.php there is the function removeProfile() I started to overwrite some fields with:

    public function removeProfile()
    {
        if (!$this->isLoaded())
        {
            return false;
        }
        $words = new MOD_words();
        $this->ProfileSummary = $words->ReplaceInMTrad("blablablabla","members.ProfileSummary", $this->getPKValue(), $this->ProfileSummary, $this->getPKValue());
        $this->WebSite = "www.i-have-left-bewelcome.com";
        $this->Accomodation = "anytime";
        $this->Organizations = $words->ReplaceInMTrad("definitively not on  BeWelcome anymore","members.Organizations", $this->getPKValue(), $this->Organizations, $this->getPKValue());

Still will have to look at translations of the profile etc...

comment:4 Changed 6 years ago by mahouni

" Current status: " Is that really the current status? I haven't found a script that changes the username etc.

What I have prepared now, is a routine that overwrites the private data (profile fields, address, name,relation comments, profile picture).

TODO: Delete gallery pictures, set location to auroville, change username, delete relations.

comment:5 Changed 6 years ago by crumbking

I'm not sure but I was reading somewhere that this is a manually process. You should write to jeanyves. Or maybe he replies here?

comment:6 Changed 6 years ago by mahouni

related tickets: #1498, #1855

comment:7 Changed 6 years ago by mahouni

crumbking:
Could you inform the BoD to give an opinion:
Do you want to allow the members to delete the profile data at the moment they confirm the deletion, (without a delay, without cron script or manually process).
There would still be some data in the db backup files though. But that's another question, I don't know how the data is archived.

Last edited 6 years ago by mahouni (previous) (diff)

comment:8 Changed 6 years ago by crumbking

It's on the agenda. (17th of Feb.)

Just copy something from JY in the mentioned BW thread here:

".... And, regardless this laws, it is a clear consenssus among BV volunteers that people must be free to leave and should have their personal data removed.

We just make a small adjustment:

  • a delay before removing the data (21 days, I assume it is the right delay) to be sure that it was not a bug or a trick done by someone stealing the password of the guy ..."

I'm not sure if this is a likely scenario?

comment:9 Changed 6 years ago by mahouni

I think the profile fields should be overwritten immediately, when a member asks for it.

TODO: Delete gallery pictures, delete empty galleries, set location to auroville, change username, delete relations.

comment:10 Changed 6 years ago by crumbking

Hi mahouni,

we talked last night about the issue. Jilrev will contact someone in France we need to keep the data for a while or not. So keep the whole thing on hold (or work on it locally till we have more info)

comment:11 Changed 5 years ago by pablobd

French authorities demand 1 year of data retention, afterwrds we can proceed to remove - see: http://www.bewelcome.org/wiki/BoD_Meeting_02.05.2013

comment:12 Changed 5 years ago by sitatara

  • Milestone changed from unassigned to backlog

comment:13 Changed 4 years ago by shevek

  • Owner changed from globetrotter_tt to shevek

Given the new terms of use having been in place for more than a year now, it's time to add the data retention.

Functionality is called through a cron job in the same way the update for the geonames database is implemented (see setup/cron_data_retention.sh).

Commit: https://gitorious.org/bewelcome/rox/commit/eeca55dc71b82ffd18cdb43f5661cc5fdaff6226

comment:14 Changed 4 years ago by shevek

  • Status changed from assigned to to_beta

comment:15 Changed 4 years ago by shevek

  • Milestone changed from backlog to 2.7

comment:16 Changed 4 years ago by leoalone

rereading i would say that comments left and received to other people should be retained in their profiles unless the poster choose to remove it explicitly.

comment:17 Changed 4 years ago by shevek

I agree. Changed the comments code so it shows comments left by 'AskToLeave?' as well. Comments aren't deleted anyway.

Commit: https://gitorious.org/bewelcome/rox/commit/44c584b18a10e56e4db76b2eafff303652f519ee

comment:18 Changed 4 years ago by shevek

  • Status changed from to_beta to testing

While the code is deployed on beta testing is not possible as it will just delete the data.

Please do a code review if you are familiar with the database structure.

comment:19 Changed 4 years ago by amnesiac84

I created the profile amnesiac84-beta and deleted it. When signing up again, it tells me the user name is taken. Don't know if it should be there, of if it means it's still in the database somewhere.

comment:20 Changed 4 years ago by leoalone

It is what is "normally" expected.
Normally an user nickname should not be reused. However there are cases when an username should be reused:

The most common case is when someone for some reason failed to confirm since entered the wrong address (can happen touchtyping on someone else keyboard with a different layout or with a defective key), or an address that is unable to read.

The second case is when someone create a testing address and then cancel it.

A third case is cybersquatting or hidden : someone registering usernames and giving email addresses of other people, not yet registered.

A fix could be that if the profile is empty, that is except for the choice of the language have been no editing, deleting it would also change the username to deleted-username-userid so freeing again the old username.

The other connected problem is about the impossibility to reuse the same email for different users: Malicious people could use this feature as a DoS against other people forbidding to register.

I suggest in case the same address is used again not to block completely but to ask, as is done for cancelling the user, if you are really sure (think about someone making account for a family member that does not use email)

comment:21 Changed 4 years ago by Tsjoek

I'm so happy to see this happening :-). I think that for BW it is important to have this done correctly. Shevek, big thanks for your work on this.

Note that the deletion should only take place one year after deletion, in accordance with French laws and from what I saw at first glance in the code, it is done that way. So it would be pretty difficult to test indeed. I had a quick look at the code and didn't see anything weird. I will try to reserve some time in the following days to check a bit more thoroughly.

comment:22 Changed 4 years ago by leoalone

you can comply with the law also by moving the data of cancelled member outside the live database, storing it off line or just mailing to the secretary of BV that will print it and store away. It is also much safer since there are no further risks of data leaks in case of programming errors.

comment:23 Changed 4 years ago by Tsjoek

or dataleaks are introduced that way because the design did no longer meet the K.I.S.S principles :-)

comment:24 Changed 4 years ago by shevek

  • Resolution set to fixed
  • Status changed from testing to closed

Enabled data retention after solving the problem with the groups showing 'login to see x more members'.

Closed as fixed after the first batch ran successfully.

Note: See TracTickets for help on using tickets.