autonomic-cooperative/civicrm-confdump
1
0
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity

Merge branch 'trunk' of ssh://git.autonomic.zone:2222/autonomic-cooperative/civicrm-confdump into trunk

Browse Source
This commit is contained in:
Cassowary Rusnov 2022-04-06 08:43:39 -07:00
commit 62099efa85
2 changed files with 11 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -5,9 +5,9 @@
- Install requirements `pip install -r requirements.txt` - Install requirements `pip install -r requirements.txt`
- `cp example.env .env` - `cp example.env .env`
- Edit `.env` and add creds for the production, test or development server you'll retrieve the data from. - Edit `.env` and add creds for the production, test or development server you'll retrieve the data from.
- Execute dump to retrieve base data: `env $(cat example.env) python ./confdump.py dump -o mydata INSTANCEPATH` where INSTANCEPATH is something like https://crm.dev.caat.org.uk/. - Execute dump to retrieve base data: `env $(cat .env) python ./confdump.py dump -o mydata INSTANCEPATH` where INSTANCEPATH is something like https://crm.dev.caat.org.uk/.
- **IMPORTANT!!** - If you sourced the data from live, you must delete the `.env` file or delete the creds from inside it. If you leave them in it will negate the whole purpose of having this conf dump utility, which is to provide a way of creating a local site without any sensitive data. - **IMPORTANT!!** - If you sourced the data from live, you must delete the `.env` file or delete the creds from inside it. If you leave them in it will negate the whole purpose of having this conf dump utility, which is to provide a way of creating a local site without any sensitive data.
- If your civicrm is in a docker container, load data into running local instance with: `confdump.py mysql -i mydata/ -p 63306` - If your civicrm is in a docker container, load data into running local instance with: `python confdump.py mysql -i mydata/ -p 63306`
- Otherwise use `confdump.py mysql -i mydata/ --host=<host> --db=<db> --user=<user> --password=<password>` - Otherwise use `confdump.py mysql -i mydata/ --host=<host> --db=<db> --user=<user> --password=<password>`
- Clear the cache in CiviCRM: - Clear the cache in CiviCRM:
* DOCKER: `make shell`, and then inside the shell `cd /app; ./vendor/bin/drush cc all` * DOCKER: `make shell`, and then inside the shell `cd /app; ./vendor/bin/drush cc all`

12
confdump.py
View File

@ -39,7 +39,8 @@ DUMP_TRIVIAL = ["FinancialType",
"CustomGroup", "CustomGroup",
"OptionGroup", "OptionGroup",
"OptionValue", "OptionValue",
"Domain"] "Domain",
"SavedSearch"]
# "ContributionPage", needs payment processors & payment_processor column formatted correctly. # "ContributionPage", needs payment processors & payment_processor column formatted correctly.
@ -64,7 +65,8 @@ LOAD_TRIVIAL = ["FinancialType",
"OptionGroup", "OptionGroup",
"OptionValue", "OptionValue",
"Domain", "Domain",
"Contact"] "Contact",
"SavedSearch"]
WEIRD_LIST = [ WEIRD_LIST = [
("civicrm_contact", "contact_sub_type"), ("civicrm_contact", "contact_sub_type"),
@ -119,6 +121,8 @@ def array_to_weird_array(val: List) -> str:
return '"\x01' + ('\x01'.join([str(x) for x in val])) + '\x01"' return '"\x01' + ('\x01'.join([str(x) for x in val])) + '\x01"'
def value_to_php_serialized(val: Any) -> str:
return "'{}'".format(mysql.escape_string(phpserialize.dumps(val).decode()).decode())
def python_value_to_sql(val: Any) -> str: def python_value_to_sql(val: Any) -> str:
""" """
@ -135,7 +139,7 @@ def python_value_to_sql(val: Any) -> str:
# weird list serialization # weird list serialization
return "'" + ','.join([str(x) for x in val]) + "'" return "'" + ','.join([str(x) for x in val]) + "'"
if (type(val) == dict): if (type(val) == dict):
return "'{}'".format(mysql.escape_string(phpserialize.dumps(val).decode()).decode()) return value_to_php_serialized(val)
return "'{}'".format(mysql.escape_string(val).decode()) return "'{}'".format(mysql.escape_string(val).decode())
@ -149,6 +153,8 @@ def dict_to_insert(table: str, objdict: Dict) -> str:
# any weird array we have to process here if there are others # any weird array we have to process here if there are others
if (table, col) in WEIRD_LIST: if (table, col) in WEIRD_LIST:
values.append(array_to_weird_array(objdict[col])) values.append(array_to_weird_array(objdict[col]))
elif table == "civicrm_saved_search" and col == "form_values":
values.append(value_to_php_serialized(objdict[col]))
else: else:
values.append(python_value_to_sql(objdict[col])) values.append(python_value_to_sql(objdict[col]))
return "REPLACE INTO {} ({}) VALUES ({});".format(table, ",".join(columns), ",".join(values)) return "REPLACE INTO {} ({}) VALUES ({});".format(table, ",".join(columns), ",".join(values))