====== Import: Which specifications has the batch process? ======


===== Manual Import =====

Please upload csv-files with addresses or target group definitions (with customer IDs or eMail addresses) to /upload/manual. 
Click on //Adresses > Lists//, choose action //Import addresses// and follow the given instructions.  You can find a manual in article //"[[en:how_to_upload_addresses]]"//.


===== Automatic Import =====

A set of standard fields is set up. Individual fields can be defined. When choosing a field description, please take into consideration that these can be used as variables in all e-mail components and that the name scheme should be standardized.

Generally, two types of automatic imports exist: 

==== Complete Import ====

The complete positive database is available and synchronized on a daily basis. The selection of target groups takes place in promio.connect or via import of target group definitions (customer ID, eMail, internal promio.connect ID, one-columned csv). 

==== Target Group Import ====

Target groups are imported as sub parts of the total database. Here the following options are available:

  * Checks for new files run daily or every five minutes.
  * Afterwards the target group is ready for further use (standard).
  * The target group is added to an existing newsletter list automatically and, thus, is ready for auto target groups afterwards. This makes managing auto campaigns (lifecycle mailings) very easy.


==== General Options ====

  * Sub clients yes/no: Should addresses for different sub clients imported in one file? This can be assigned with b2b_user_id. 
  * Time of checks for new files (only for daily checks, at the hour 0-24).



==== Requirements list for automatic import ====

|**Client**|client number|
|**Test file**|This file must correspond exactly to the files that will be uploaded later, regarding charset, formatting (csv, bz2, Z), data structure (number + order of columns, separator, field delimiter).|
|**E-mail address/es**|For receiving of status mails (logfiles) of all import processes.|
|**Address management feature**|custID OR mail; can be chosen differing from client configuration.|
|**Time of import**|Process is checking whether a file has been uploaded: daily at x o'clock OR every 5 minutes.|
|**Creating a new target group**|Per import OR adding to a auto target group.|
|**If auto target group**|1) Adding OR replacing of the auto target group - possible methods: \\ - replace \\ - replaceNoResubscribes \\ - add \\ - addNoResubscribes \\ \\ 2) if auto target group is replaced: \\ Unsubscribing of all users that are not included in the current file?* \\ * Please note: this is not recommended and only possible if user data are synchronized completely and every night because users will be added to the blacklist.|


==== Import Structure ====

File structure / Nomenclature:

  * Format: .csv
  * Compression (optional): bzip2, gzip with the ending .bz2, .Z
  * Encoding (optional): DES openssh
  * File name:
    * [SENDERID]_[NAME]_[CHARSET]_[IMPORTTEMPLATEID]_YYYYMMDDHHMMSS.csv
      * SENDERID: ID of the client to which the file is imported
      * NAME: Target group description that is used as the name in the system after import. Please do not use special characters or underscores.
      * CHARSET: [utf8|latin]
      * IMPORTTEMPLATEID: Integer, a variable that is provided by promio.net and that defines all information for the defined filetype (columns and type assignment always have to be constant)
  * Validity: The following things are automatically validated: 
    * Filename on correct syntax
    * Number of lines in the file vs. number of lines after import into a temporary table
    * E-Mail on correct syntax. Duplicates will be ignored.
  * Logfile: After the import a logfile is supplied in the system and, in addition, an e-mail with error and warning information is sent to an defined distribution list.
  * Files have to be uploaded into the directory /upload/auto/ of promio.connect SFTP server.



===== Standard field structure (internal) =====


This is the standard file structure which can be extended for individual purposes.


|**Fieldname**|**Fieldtype**|**Standard**|**Comment**|
|b2b_user_id|int(8)|0|internal client ID: [xxxx]|
|id|int(9)| |internal customer ID. System field of promio.connect.|
|mail|varchar(255)| |e-mail address| 	 
|nick|varchar(255)| |nickname|
|vorname|varchar(255)| |first name| 	 	 
|name|varchar(255)| |surname| 
|gender|tinyint(1)|0|1=male, 2=female|
|mailerror|tinyint(1)|0|Internal last bounce status. System field of promio.connect.|
|unsubscribe|tinyint(1)|0|Internal unsubscribe status. System field of promio.connect.|
|source|tinyint(1)|0|Internal import marker. System field of promio.connect.|
|strasse|varchar(255)| |street| 	 
|plz |varchar(255)| |postal code| 	 	 
|ort|varchar(255)| |place| 	 
|custID|varbinary(255)| |external customer ID| 	 	
|country|varchar(255)| |de, at, ch, ...|
|reg_datetime|datetime|0000-00-00 00:00:00|Time of registration|
|reg_ip|varchar(255)| |Registration IP|
|tsCreated|timestamp|0000-00-00 00:00:00|Timestamp of creation of data set. System field of promio.connect.|
|tsLastUpdate|timestamp|on update CURRENT_TIMESTAMP|Timestamp of last update of data set. System field of promio.connect.|
|agr|tinyint(3)|0|Advertisement agreement (1=given)|
|tel|varchar(255)| |phone number| 	 	 
|birthday|date|0000-00-00|birthday| 	 
|pm_image|varchar(255)| |Link to an image of person (address research). Can be filled manually.| 	 	 
|pm_reputation_score|tinyint(1)| |Internal reputation score. System field of promio.connect.| 	 
|special_1|varchar(255)| |Free variable for future use|
|special_2|varchar(255)| |Free variable for future use|
|special_3|varchar(255)| |Free variable for future use|
|special_4|varchar(255)| |Free variable for future use|
|special_5|varchar(255)| |Free variable for future use|
|special_6|varchar(255)| |Free variable for future use|
|special_7|varchar(255)| |Free variable for future use|
|special_8|varchar(255)| |Free variable for future use|
|special_9|varchar(255)| |Free variable for future use|
|special_10|varchar(255)| |Free variable for future use|
|html_version|tinyint(1)| |format of view of email|
|test_flag|tinyint(1)|0|internal marker for test user|
|test_letter_id|int(6)|0|internal marker for test user|


===== Specifications personal data for import =====


|**Fieldname**|**Fieldtype**|**Standard**|**Comment**|
|b2b_user_id|int(8)|0|internal client ID: [xxxx]|
|id|int(9)| |internal customer ID|
|mail|varchar(255)| |e-mail address | 	 
|nick|varchar(255)| |nickname|
|vorname|varchar(255)| |first name| 	 	 
|name|varchar(255)| |surname| 
|gender|tinyint(1)|0|1=male, 2=female|
|strasse|varchar(255)| |street| 	 	
|plz|varchar(255)| |postal code| 	 	 
|ort|varchar(255)| |city| 	 
|custID|varbinary(255)| |external customer ID| 	 	
|country|varchar(255)| |de, at, ch, ...|
|reg_datetime|datetime|0000-00-00 00:00:00|Registration date|
|reg_ip|varchar(255)| |Registration IP|
|agr|tinyint(3)|0|Advertisement agreement (1=given)|
|tel|varchar(255)| |phone number | 	 	 
|birthday|date|0000-00-00|birthday| 	 
|special_1|varchar(255)| |Free variable for future use|
|special_2|varchar(255)| |Free variable for future use|

===== Automatic delivery after importing a target group =====

After the automatic import of a target group, you can trigger additional tasks by invoking an API call. A common application is the automatic set-up of an e-mail with subsequent delivery. 

This requires configuring an automatic import process through promio.net support. 

To do so, you must provide the two files //user.csv// und //action.json// in a ZIP archive. \\ 
This ZIP file must be named according to the agreed upon naming scheme and provided in the SFTP folder /upload/auto for automatic processing. 

==== user.csv ====

Contains the data configured for automatic import.

<code>
adress1@mail.de
adress2@mail.de
</code>


==== action.json ====

Contains campaign and target group information in JSON format. \\
These are used after the automatic import for the execution of REST API calls.


=== Maximum specification with all necessary and optional properties ===
<code JavaScript>
{
 "clientId": 10105,
 "campaign": { 
   "name": "My campaign", 
   "code": "ABC-123-DEF",
   "clientId": 12125,
   "newsletterId": 82384
 },
 "message": { 
     "templateId": 2861485, 
     "format": "html",
     "name": "My message", 
     "sendFromId": 12944,
     "sendFromName": "promio.net newsletter",
     "locale": "de_DE",
     "creative": {
       "content": "<html>\n <head>\n   <title>My first message</title>\n </head>\n <body>\n   <p>Hello World!</p>\n </body>\n</html>\n",
       "subject": "My subject"
     },
     "text": {
       "content": "Hello World.\n"
     }
 },
 "targetGroup": {
   "filter": {
     "includes": [
       315089
     ],
     "excludes": [
       {
         "type": "list",
         "id": 100898
       }
     ]
   },
   "limit": {
     "amount": 50000,
     "type": "mostActive"
   }
 },
 "delivery": {
   "schedule": {
     "scheduledForDeliveryAt": "2025-03-01 12:00" 
   },
   "start": {
     "mailsPerHourLimit": 5000,
     "resetStatistics": true,
     "refreshAutoTargetGroup": true
   },
   "sendTest": {
     "recipients": [
       {
         "custId": "abc-1234",
         "mail": "max.mustermann@musterdomain.de",
         "lastName": "Mustermann",
         "firstName": "Max",
         "gender": 1,
         "street": "Musterstraße 4",
         "city": "Musterstadt",
         "countryCode": "DE",
         "postcode": "12345",
         "birthday": "1986-02-12",
         "phoneNumber": 4912345678
       }
     ]
   }
 }
}
</code>

=== Required fields and instructions ===

  * //clientId//:
    * necessary. Enter ID of the client.
  * //campaign//:
    * required. Required field within: //name//
    * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/campaign-write/operation/createCampaign|REST API - create campaign]]
  * //message//:
    * required
    * if //templateId// exists
      * message is built from a template
      * required: //name//
    * if //templateId// not exists
      * required: //name//
      * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/Create-or-update-messages/operation/createMessage|REST API - create message]]
    * //creative//:
      * required for messages that are not created by a template
      * required fields: //content//, //subject//
    * //text//:
      * optional. Only necessary for a different text part. If //text// is not specified, the text part is automatically generated from the HTML of the multipart e-mail. 
  * //targetGroup//
      * optional. Only necessary for setting //filter// and //limit//.
      * Target group ID is assigned automatically.
      * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/Target-message-audience/operation/assignTargetGroupToMessage|REST API - assign target group]]
  * //delivery//
      * optional.  Only necessary if delivery is to be scheduled or started.
      * settings for delivery
      * //schedule//
        * for scheduling the campaign
        * required: //scheduledForDeliveryAt//
        * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/Manage-campaign-delivery/operation/scheduleCampaign|REST API - schedule delivery]]
      * //start//
        * delivery is only started if //start// is set.
        * no required fields
        * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/Manage-campaign-delivery/operation/sendCampaign|REST API - start delivery]]
      * //sendTest//
        * optional. Send a test mail.
        * Docs: [[https://api.promio-connect.com/docs/3.1/#tag/Testing-messages/operation/sendTestMail|REST API - send test mail]]

**Important:** \\
The source code in //creative// and //text// must be "prepared" for json. 
The following (external) website provides information on how to escape, as well as an option to have source text escaped automatically: https://www.freeformatter.com/json-escape.html

=== Examples ===

Example of creating a message via HTML source code (incl. start delivery):
<code JavaScript>
{
 "clientId": 10105,
  "campaign": {
    "name": "My campaign"
  },
  "message": {
    "format": "html",
    "name": "My message",
    "sendFromId": 12944,
    "sendFromName": "promio.net newsletter",
    "creative": {
      "content": "<html>\n  <head>\n    <title>My first message</title>\n </head>\n <body>\n   <p>Hello World!</p>\n </body>\n</html>\n",
      "subject": "My subject"
    }
  },
  "delivery": {
    "schedule": {
      "scheduledForDeliveryAt": "2025-03-01 12:00"
    },
    "start": {
    }
  }
}
</code>

Example of creating a message via template (without start delivery):
<code JavaScript>
{
 "clientId": 10105,
  "campaign": {
    "name": "My campaign"
  },
  "message": {
    "name": "My message",
    "templateId": 2861485
  }
}
</code>


{{tag>imports batch processes automatical specifications nomenclature record_descriptions }}