Generating list of unique email addresses fails with StackOverflow

```kotlin
            context("Unique email generation") {
                fun createFaker(): Faker {
                    val cfg = fakerConfig { uniqueGeneratorRetryLimit = 10_000 }
                    val f = Faker(cfg)
                    f.unique.configuration { enable(f::internet) }
                    return f
                }

                it("should generate a list of unique emails") {
                    val f = createFaker()
                    List(100) { f.internet.email() }
                    // TODO: can add assertion,
                    // but we're really just testing absense of StackOverflow error
                }
            }
```

```
java.util.regex.PatternSyntaxException: Stack overflow during pattern compilation near index 13
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
             ^
	at java.base/java.util.regex.Pattern.error(Pattern.java:2028)
	at java.base/java.util.regex.Pattern.<init>(Pattern.java:1432)
	at java.base/java.util.regex.Pattern.compile(Pattern.java:1069)
	at kotlin.text.Regex.<init>(Regex.kt:90)
	at io.github.serpro69.kfaker.FakerService.resolveExpression-goSy63c(FakerService.kt:516)
	at io.github.serpro69.kfaker.FakerService.resolve(FakerService.kt:435)
	at io.github.serpro69.kfaker.provider.YamlFakeDataProvider.returnOrResolveUnique$lambda$2(YamlFakeDataProvider.kt:227)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:74)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
```

```
java.util.regex.PatternSyntaxException: Stack overflow during pattern compilation near index 23
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
                       ^
	at java.base/java.util.regex.Pattern.error(Pattern.java:2028)
	at java.base/java.util.regex.Pattern.<init>(Pattern.java:1432)
	at java.base/java.util.regex.Pattern.compile(Pattern.java:1069)
	at kotlin.text.Regex.<init>(Regex.kt:90)
	at io.github.serpro69.kfaker.FakerService.resolveExpression-goSy63c(FakerService.kt:516)
	at io.github.serpro69.kfaker.FakerService.resolve(FakerService.kt:435)
	at io.github.serpro69.kfaker.provider.YamlFakeDataProvider.returnOrResolveUnique$lambda$2(YamlFakeDataProvider.kt:227)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:74)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue$lambda$0(AbstractFakeDataProvider.kt:77)
	at io.github.serpro69.kfaker.provider.AbstractFakeDataProvider.resolveUniqueValue(AbstractFakeDataProvider.kt:155)
```

I think we're trying to create unique domain, of which there are only 2 for this use-case: `example` and `test`, and after they're exhausted, we get SO due to a high retry-limit (`10_000`):

```diff
diff --git a/core/src/main/kotlin/io/github/serpro69/kfaker/FakerService.kt b/core/src/main/kotlin/io/github/serpro69/kfaker/FakerService.kt
index 0c1c65a3..3a7eda19 100644
--- a/core/src/main/kotlin/io/github/serpro69/kfaker/FakerService.kt
+++ b/core/src/main/kotlin/io/github/serpro69/kfaker/FakerService.kt
@@ -629,6 +629,7 @@ class FakerService {
         //
         // #\{(?!\d)(?:(Creature|Games|(Bird|Cat|Dog))\.)?((?![A-Z]\p{L}*\.).*?)\}
         val lexpr = Regex("""#\{(?!\d)(?i:($primary|($secondary))\.)?((?![A-Z]\p{L}*\.).*?)\}""")
+        println(lexpr)
         // https://regex101.com/r/I8gG7M/1
         //
         // #\{(?!\d)(?!(?i:Creature|(?i:Bird|Cat|Dog))\.)(?:([A-Z]\p{L}+())\.)?(.*?)\}
diff --git a/core/src/main/kotlin/io/github/serpro69/kfaker/provider/Internet.kt b/core/src/main/kotlin/io/github/serpro69/kfaker/provider/Internet.kt
index af9d4c4d..5a1c4317 100644
--- a/core/src/main/kotlin/io/github/serpro69/kfaker/provider/Internet.kt
+++ b/core/src/main/kotlin/io/github/serpro69/kfaker/provider/Internet.kt
@@ -25,7 +25,7 @@ class Internet internal constructor(fakerService: FakerService, private val name
         val name: () -> String = {
             prepare(nameProvider.lastName().split(" ").first(), fakerService.faker.config)
         }
-        return domain?.let {
+        val d = domain?.let {
             domain
                 .split(".")
                 .map { domainPart -> prepare(domainPart, fakerService.faker.config) }
@@ -42,6 +42,8 @@ class Internet internal constructor(fakerService: FakerService, private val name
                     .also { if (subdomain) it.add(0, prepare(name(), fakerService.faker.config)) }
                     .joinToString(".")
             }
+        println(d)
+        return d
     }

     /** Returns a random IPv4 address */
```

```
io.github.serpro69.kfaker.provider.InternetTest > Internet provider > Unique email generation > io.github.serpro69.kfaker.provider.InternetTest.should generate a list of unique emails STANDARD_OUT
    #\{(?!\d)(?i:(Name|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
dach.example
    dach.example
#\{(?!\d)(?i:(Name|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Name|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
lebsack.test
    lebsack.test
#\{(?!\d)(?i:(Name|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Name|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Internet|())\.)?((?![A-Z]\p{L}*\.).*?)\}
```

In fact, if I use the default retry limit of 100, I get `io.github.serpro69.kfaker.exception.RetryLimitException: Retry limit of 100 exceeded`

---

Yeah, so this is the "root cause", since we use safeDomainSuffix in email generation:

```kotlin
List(100) { f.internet.unique.safeDomainSuffix() }
```

The same would fail for any other provider/function that does not have enough unique values:

```koltin
List(100) { f.address.unique.cityPrefix() }
```

```
    #\{(?!\d)(?i:(Address|())\.)?((?![A-Z]\p{L}*\.).*?)\}
#\{(?!\d)(?i:(Address|())\.)?((?![A-Z]\p{L}*\.).*?)\}
    #\{(?!\d)(?i:(Address|())\.)?((?![A-Z]\p{L}*\.).*?)\}

io.github.serpro69.kfaker.provider.InternetTest > Internet provider > Unique email generation > io.github.serpro69.kfaker.provider.InternetTest.should generate a list of unique emails FAILED
    java.lang.StackOverflowError
        at _COROUTINE._BOUNDARY._(CoroutineDebugging.kt:42)
        at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:101)
        at io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor.intercept(InvocationTimeoutInterceptor.kt:42)
        at io.kotest.engine.test.TestInvocationInterceptor$runBeforeTestAfter$wrappedTest$1$1.invokeSuspend(TestInvocationInterceptor.kt:70)
        at io.kotest.engine.test.TestInvocationInterceptor$intercept$2$1.invokeSuspend(TestInvocationInterceptor.kt:36)
        at io.kotest.mpp.ReplayKt.replay(replay.kt:15)
        at io.kotest.engine.test.TestInvocationInterceptor$intercept$2.invokeSuspend(TestInvocationInterceptor.kt:32)
        at io.kotest.engine.test.TestInvocationInterceptor.intercept(TestInvocationInterceptor.kt:31)
        at io.kotest.engine.test.TestCaseExecutor$execute$3$1.invokeSuspend(TestCaseExecutor.kt:100)
        at io.kotest.engine.interceptors.MarkAbortedExceptionsAsSkippedTestInterceptor.intercept(MarkAbortedExceptionsAsSkippedTestInterceptor.kt:23)
        at io.kotest.engine.test.TestCaseExecutor$execute$3$1.invokeSuspend(TestCaseExecutor.kt:100)

```

I don't know if we can handle every single case or should just "outsource" the decision to users on what to generate as unique values and what not (this behavior is documented - https://github.com/serpro69/kotlin-faker/blob/ea4e361ce800ade4ad9dd7b368b27346b644df02/docs/wiki/unique-generator.md?plain=1#L180-L199 )
But for "internally-called functions" (i.e. email needing `safeDomainSuffix` which only has 2 values - it should be handled by the faker)

	!!! info
	Even if `uniqueGeneratorRetryLimit` is set to a very high number, one still needs to keep in mind the dataset size for a given faker function to avoid unexpected exceptions when generating unique values.
	<br>
	For example, a `city_prefix` param in the `address` category only has 7 values:

	??? example "address.yml"
	```yaml
	en:
	faker:
	address:
	city_prefix: [North, East, West, South, New, Lake, Port]
	# rest of the address.yml dict file
	```

	Therefore, the initial pool size of unique values is quite small and will be exhausted very quickly, so the retry limit might need to be set to a higher than default value.
	<br>
	And obviously if one tries to generate 8 unique city prefixes a `RetryLimitException` will be thrown sooner or later, irrespective of what value is set for the `uniqueGeneratorRetryLimit` config property.
	<br>
	<br>
	A reference list of available Data Generators with their corresponding yml dictionary data can be found on the [Data Generators](./data-providers.md) wiki page.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Generating list of unique email addresses fails with StackOverflow #284

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Uh oh!

Generating list of unique email addresses fails with StackOverflow #284

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions