Univention Bugzilla – Bug 42503
ucs-school-import: ImportUser.make_email() does not honor config[mandatory_attributes]
Last modified: 2017-10-17 12:47:54 CEST
ImportUser.make_email() raises an exception if no email address is either supplied in the input data or can be created from a scheme. It ignores config[mandatory_attributes]: if there is no "email" in the list, no exception should be raised, but the attribute should be left empty.
I set "How will those affected feel about the bug?" to "nuisance", because a simple workaround exists: supply mailPrimaryAddress="" for all users to get empty email addresses.
I will set interim keyword later on.
When fixing this bug, please check the ucs-test scripts for then-unneeded handler_set(['mail/hosteddomains=..']).
After fixing this bug, please revert r74028.
(In reply to Daniel Tröder from comment #4) > After fixing this bug, please revert r74028. Fixed last commit. So a simple revert will not work.
r74469: code and advisory r74471: reverted workarounds in ucs-test-ucsschool
r74475: post-autopep8 fixes, whitespace-only
OK: YAML OK: I could import a user which has no email set REOPEN: But after the import always this traceback appears: # cat foo.csv "Schulen","Benutzertyp","Vorname","Nachname","Klassen","Beschreibung","Telefon","recordUID" "oldschool","student","Ingward","Heckmann","oldschool-1,oldschool-a","A student.","+99-218-122014","ingward" # /usr/share/ucs-school-import/scripts/ucs-school-user-import -i foo.csv --sourceUID=foo -u teacher -s oldschool ------ UCS@school import tool starting ------ Reading configuration from '/usr/share/ucs-school-import/configs/global_defaults.json'... Reading configuration from '/var/lib/ucs-school-import/configs/global.json'... Reading configuration from '/usr/share/ucs-school-import/configs/user_import_defaults.json'... Reading configuration from '/var/lib/ucs-school-import/configs/user_import.json'... 2016-11-16 03:16:49 INFO cmdline.main:111 ------ UCS@school import tool configured ------ 2016-11-16 03:16:49 INFO cmdline.main:112 Used configuration files: ['/usr/share/ucs-school-import/configs/global_defaults.json', '/var/lib/ucs-school-import/configs/global.json', '/usr/share/ucs-school-import/configs/user_import_defaults.json', '/var/lib/ucs-school-impo rt/configs/user_import.json']. 2016-11-16 03:16:49 INFO cmdline.main:113 Using command line arguments: {'input': {'filename': 'foo.csv'}, 'school': 'oldschool', 'user_role': 'teacher', 'sourceUID': 'foo'} 2016-11-16 03:16:49 INFO cmdline.main:114 Configuration is: {u'activate_new_users': {u'default': True}, u'classes': {}, u'csv': {u'header_lines': 1, u'incell-delimiter': {u'default': u','}, u'mapping': {u'Nachname': u'lastname', u'Vorname': u'firstname'}}, u'dry_run': False, u'factory': u'ucsschool.importer.default_user_import_factory.DefaultUserImportFactory', u'input': {u'filename': 'foo.csv', u'type': u'csv'}, u'logfile': u'/var/log/univention/ucs-school-import.log', u'maildomain': None, u'mandatory_attributes': [u'firstname', u'lastname', u'name', u'school'], u'no_delete': False, u'output': {u'new_user_passwords': None, u'user_import_summary': u'/var/lib/ucs-school-import/user_import_summary_%Y-%m-%d_%H:%M:%S.csv'}, u'password_length': 15, u'scheme': {u'email': u'<firstname>[0].<lastname>@<maildomain>', u'recordUID': u'<username>', u'username': {u'allow_rename': False, u'default': u'<:umlauts><firstname>[0].<lastname><:lower>[COUNTER2]'}}, u'school': 'oldschool', u'sourceUID': 'foo', u'tolerate_errors': 0, u'user_deletion': {u'delete': True, u'expiration': 0}, u'user_role': 'teacher', u'verbose': True} 2016-11-16 03:16:49 INFO cmdline.do_import:94 ------ Starting mass import... ------ 2016-11-16 03:16:49 WARNING utils.stopped_notifier:294 Stopping univention-directory-notifier 2016-11-16 03:16:54 INFO utils._run:288 Stopping univention-directory-notifier daemon: . ok: down: univention-directory-notifier: 0s done. 2016-11-16 03:16:54 INFO utils.stopped_notifier:306 univention-directory-notifier stopped 2016-11-16 03:16:54 INFO mass_import.import_users:94 ------ Importing users... ------ 2016-11-16 03:16:54 INFO user_import.read_input:75 ------ Starting to read users from input data... ------ 2016-11-16 03:16:54 DEBUG base_reader.next:73 Input 2: ['oldschool', 'student', 'Ingward', 'Heckmann', 'oldschool-1,oldschool-a', 'A student.', '+99-218-122014', 'ingward'] -> {u'Schulen': u'oldschool', u'Benutzertyp': u'student', u'recordUID': u'ingward', u'Nachname': u' Heckmann', u'Telefon': u'+99-218-122014', u'Beschreibung': u'A student.', u'Vorname': u'Ingward', u'Klassen': u'oldschool-1,oldschool-a'} 2016-11-16 03:16:54 DEBUG csv_reader.map:195 ImportTeacher(name=None, school=None, dn=None) attributes={'$dn$': None, 'display_name': u'Ingward Heckmann', 'record_uid': None, 'firstname': u'Ingward', 'lastname': u'Heckmann', 'type_name': 'Teacher', 'school': None, 'name': None, 'disabled': None, 'email': None, 'birthday': None, 'type': 'importTeacher', 'schools': [], 'password': None, 'source_uid': None, 'school_classes': {}, 'objectType': 'users/user'} udm_properties={} action=None 2016-11-16 03:16:54 WARNING import_user.format_from_scheme:681 Created empty 'recordUID' from scheme '<username>' and input data {'$dn$': None, u'firstname': u'Ingward', u'lastname': u'Heckmann', 'disabled': None, 'birthday': None, 'schools': [], 'password': None, 'source _uid': None, 'name': None, 'school': None, 'display_name': u'Ingward Heckmann', 'record_uid': None, 'type_name': 'Teacher', 'school_classes': {}, 'objectType': 'users/user', 'type': 'importTeacher', 'email': None}. 2016-11-16 03:16:54 INFO user_import.read_input:79 Done reading 1. user: ImportTeacher(name=None, school=None, dn=None) 2016-11-16 03:16:54 INFO user_import.read_input:87 ------ Read 1 users from input data. ------ 2016-11-16 03:16:54 INFO user_import.detect_users_to_delete:210 ------ Detecting which users to delete... ------ 2016-11-16 03:16:54 DEBUG user_import.detect_users_to_delete:250 users_to_delete=[] 2016-11-16 03:16:54 INFO user_import.delete_users:265 ------ Deleting 0 users... ------ 2016-11-16 03:16:54 INFO user_import.delete_users:281 ------ Deleted 0 users. ------ 2016-11-16 03:16:54 INFO user_import.create_and_modify_users:101 ------ Creating / modifying users... ------ 2016-11-16 03:16:54 DEBUG user_import.create_and_modify_users:106 Creating / modifying user ImportTeacher(name=None, school=None, dn=None)... 2016-11-16 03:16:54 DEBUG base.get_only_udm_obj:869 Getting ImportTeacher UDM object by filter: (&(objectClass=ucsschoolType)(ucsschoolSourceUID=foo)(ucsschoolRecordUID=)) 2016-11-16 03:16:54 WARNING import_user.format_from_scheme:681 Created empty 'recordUID' from scheme '<username>' and input data {'$dn$': None, u'firstname': u'Ingward', u'lastname': u'Heckmann', 'disabled': None, 'birthday': None, 'schools': [], 'password': None, 'source _uid': 'foo', 'name': None, 'school': None, 'display_name': u'Ingward Heckmann', 'record_uid': u'', 'type_name': 'Teacher', 'school_classes': {}, 'objectType': 'users/user', 'type': 'importTeacher', 'email': None}. 2016-11-16 03:16:54 INFO user_import.create_and_modify_users:121 Adding ImportTeacher(name='i.heckmann5', school='oldschool', dn='uid=i.heckmann5,cn=lehrer,cn=users,ou=oldschool,dc=school,dc=local', old_dn=None) (source_uid:foo record_uid:) attributes={'$dn$': 'uid=i.hec kmann5,cn=lehrer,cn=users,ou=oldschool,dc=school,dc=local', 'display_name': 'Ingward Heckmann', 'record_uid': u'', 'firstname': 'Ingward', 'lastname': 'Heckmann', 'type_name': 'Teacher', 'school': 'oldschool', 'name': 'i.heckmann5', 'disabled': 'none', 'email': None, 'birt hday': None, 'type': 'importTeacher', 'schools': ['oldschool'], 'password': 'ENQMX9duK0pb1B,', 'source_uid': 'foo', 'school_classes': {}, 'objectType': 'users/user'} udm_properties={'overridePWHistory': '1', 'overridePWLength': '1'}... 2016-11-16 03:16:54 INFO pyhooks_loader.get_plugins:53 Searching for plugins in: /usr/share/ucs-school-import/pyhooks... 2016-11-16 03:16:54 INFO pyhooks_loader.get_plugins:61 Found plugins: [] 2016-11-16 03:16:54 INFO import_user.setup_pyhooks:730 Registered hooks: {}. 2016-11-16 03:16:54 DEBUG base.call_hooks:373 /usr/share/ucs-school-import/hooks/user_create_pre.d not found or empty. 2016-11-16 03:16:54 DEBUG base.get_only_udm_obj:869 Getting ImportTeacher UDM object by filter: username=i.heckmann5 2016-11-16 03:16:54 INFO base.create_without_hooks:435 Creating ImportTeacher(name='i.heckmann5', school='oldschool', dn='uid=i.heckmann5,cn=lehrer,cn=users,ou=oldschool,dc=school,dc=local', old_dn=None) 2016-11-16 03:16:54 DEBUG base.cache:210 Initializing ('School', ('name', 'oldschool')) 2016-11-16 03:16:54 DEBUG base.get_udm_object:666 Getting School UDM object by dn: ou=oldschool,dc=school,dc=local 2016-11-16 03:16:54 DEBUG base.cache:210 Initializing ('Group', ('name', 'Domain Users oldschool'), ('school', 'oldschool')) 2016-11-16 03:16:54 DEBUG base.call_hooks:373 /usr/share/ucs-school-import/hooks/group_create_pre.d not found or empty. 2016-11-16 03:16:54 DEBUG base.get_only_udm_obj:869 Getting Group UDM object by filter: name=Domain Users oldschool 2016-11-16 03:16:54 DEBUG base.cache:210 Initializing ('Group', ('name', 'lehrer-oldschool'), ('school', 'oldschool')) 2016-11-16 03:16:54 DEBUG base.get_only_udm_obj:869 Getting Group UDM object by filter: name=lehrer-oldschool 2016-11-16 03:16:54 DEBUG base.from_dn:852 Looking up School with dn 'ou=oldschool,dc=school,dc=local' 2016-11-16 03:16:54 DEBUG base.get_all:737 Getting all AnyComputer of oldschool with filter 'univentionService=Windows Profile Server' 2016-11-16 03:16:55 INFO base.create_without_hooks:457 ImportTeacher(name='i.heckmann5', school='oldschool', dn='uid=i.heckmann5,cn=lehrer,cn=users,ou=oldschool,dc=school,dc=local') successfully created 2016-11-16 03:16:55 DEBUG base.get_only_udm_obj:869 Getting ImportTeacher UDM object by filter: (&(objectClass=ucsschoolType)(ucsschoolSourceUID=foo)(ucsschoolRecordUID=)) 2016-11-16 03:16:55 WARNING utils.stopped_notifier:316 Starting univention-directory-notifier 2016-11-16 03:16:55 INFO utils._run:288 Starting Univention Directory Notifier daemon. ok: run: univention-directory-notifier: (pid 20322) 0s, normally down done. 2016-11-16 03:16:55 INFO utils.stopped_notifier:323 univention-directory-notifier started 2016-11-16 03:16:55 ERROR cmdline.main:137 Outer Exception catcher: noObject('No user with source_uid=foo and record_uid= found.',) Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 117, in main self.do_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 95, in do_import importer.mass_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 70, in mass_import self.import_users() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 99, in import_users user_import.create_and_modify_users(imported_users) File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/user_import.py", line 131, in create_and_modify_users success = user.create(lo=self.connection) File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 150, in create return super(ImportUser, self).create(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 429, in create self.call_hooks('post', 'create') File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 124, in call_hooks user = self.get_by_import_id(self._lo, self.source_uid, self.record_uid) File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 166, in get_by_import_id raise noObject("No user with source_uid={0} and record_uid={1} found.".format(source_uid, record_uid)) noObject: No user with source_uid=foo and record_uid= found. → I am not sure why this happens, everything else seems to work. Please decide what to do with it.
(In reply to Florian Best from comment #8) > REOPEN: But after the import always this traceback appears: > > # cat foo.csv > "Schulen","Benutzertyp","Vorname","Nachname","Klassen","Beschreibung", > "Telefon","recordUID" > "oldschool","student","Ingward","Heckmann","oldschool-1,oldschool-a","A > student.","+99-218-122014","ingward" > > > # /usr/share/ucs-school-import/scripts/ucs-school-user-import -i foo.csv > --sourceUID=foo -u teacher -s oldschool [..] > u'scheme': {u'email': u'<firstname>[0].<lastname>@<maildomain>', > u'recordUID': u'<username>', > u'username': {u'allow_rename': False, > u'default': > u'<:umlauts><firstname>[0].<lastname><:lower>[COUNTER2]'}}, [..] > noObject: No user with source_uid=foo and record_uid= found. > > → I am not sure why this happens, everything else seems to work. Please > decide what to do with it. There is already a in separate bug for this problem: Bug #42137.
Yes!
This introduces a syntax error in r74475 causing: 2016-12-01 18:13:09 ERROR cmdline.main:137 Outer Exception catcher: TypeError('__init__() takes at least 3 arguments (1 given)',) Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 117, in main self.do_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 95, in do_import importer.mass_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 70, in mass_import self.import_users() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 99, in import_users user_import.create_and_modify_users(imported_users) File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/user_import.py", line 151, in create_and_modify_users sys.exc_info()[2] TypeError: __init__() takes at least 3 arguments (1 given)
And another one in: ucs-school-import/modules/ucsschool/importer/reader/csv_reader.py
r74926: fix syntax error from r74475
OK: syntax errors
UCS@school 4.1 R2 v9 has been released. http://docs.software-univention.de/changelog-ucsschool-4.1R2v9-de.html